Work order auditing method and device and training method of work order auditing large model

Abstract

The application is suitable for the technical field of computer application, and provides a work order auditing method and device and a training method of a work order auditing large model. The method comprises the following steps: obtaining a target application program corresponding to a correction work order submitted by a user; determining translation correction feedback information corresponding to the correction work order according to the correction work order; determining whether the translation correction feedback information meets preset language specification conditions and preset relevance conditions through a preset work order auditing large model; and in the case that the translation correction feedback information meets the preset language specification conditions and the preset relevance conditions, determining that the correction work order passes the audit. Thus, the translation correction feedback information of the correction work order submitted by the user is subjected to language specification checking and relevance checking through the preset work order auditing large model, so as to audit the correction work order, thereby improving the auditing efficiency of the correction work order.

Description

Technical Field

[0001] This application belongs to the field of computer application technology, and in particular relates to a work order review method, device, electronic device, computer-readable storage medium, computer program product, and training method for a large work order review model. Background Technology

[0002] With the rapid development of globalized digital products, many applications are widely deployed and used in numerous countries and regions. Due to language differences between countries and regions, product interfaces and operation instructions need to be translated into local languages ​​to meet user needs. However, machine translation or human translation inevitably leads to mistranslations or cultural incompatibility, affecting user experience. To address this, many applications allow users to submit translation correction requests to continuously optimize translation quality. However, in practice, user-submitted content may contain non-standard terminology or be irrelevant to the translation issue. Without review, this can not only affect platform content security but also interfere with subsequent processing. Therefore, an effective review mechanism must be established to screen and filter user-submitted correction requests.

[0003] In related technologies, manual review is the primary method for verifying translation error correction requests submitted by users. However, with the increase in supported languages ​​and the expansion of user feedback, manual review has become inefficient. Summary of the Invention

[0004] The purpose of this application is to provide a work order review method, device, electronic device and computer-readable storage medium, computer program product and training method for a large work order review model, which can solve the problem of low work order review efficiency caused by using manual review to review translation correction work orders submitted by users in related technologies.

[0005] In a first aspect, embodiments of this application provide a work order review method, including: obtaining a correction work order corresponding to a target application submitted by a user; determining the translation correction feedback information corresponding to the correction work order based on the correction work order; determining whether the translation correction feedback information meets preset language standard conditions and preset relevance conditions through a preset work order review model; and determining that the correction work order review is passed if the translation correction feedback information meets the preset language standard conditions and preset relevance conditions.

[0006] In one possible implementation of the first aspect, the aforementioned translation error correction feedback information includes translation error correction text and translation interface images; the aforementioned preset language standardization conditions include at least one blacklist field; the aforementioned preset relevance conditions include a relevance threshold; and the aforementioned determination of whether the translation error correction feedback information meets the preset language standardization conditions and preset relevance conditions through a preset work order review model includes:

[0007] Input the translation error correction feedback information into the preset work order review model; By using a pre-set work order review model, it can be determined whether the translated error-correction text includes blacklist fields. If the translation correction text does not include any blacklist field, it is determined that the translation correction feedback information meets the preset language specification conditions; By using a pre-set work order review model, the relevance between the translated error correction text and the translation interface images is determined. If the relevance is greater than or equal to the relevance threshold, the translation error correction feedback information is determined to meet the preset relevance condition.

[0008] Optionally, in another possible implementation of the first aspect, determining whether the translated error-correction text includes a blacklist field includes: The translated text undergoes word segmentation to generate a word segmentation sequence, wherein the word segmentation sequence includes at least one word. Each word segment is matched against each blacklist field to determine the matching degree between each word segment and each blacklist field. If any word segment matches any blacklist field with a degree greater than a preset matching threshold, the translated text to be corrected includes the blacklist field. If the matching degree between all word segments and each blacklist field is less than or equal to the preset matching degree threshold, it is determined that the translated and corrected text does not include any blacklist field.

[0009] Optionally, in another possible implementation of the first aspect, determining the relevance between the translated error-correction text and the translation interface image includes: Extract text from images on the translation interface to generate image text; Feature extraction is performed on the image text to determine the image text features corresponding to the image text; Feature extraction is performed on the translation correction text to determine the features of the correction text corresponding to the translation correction text; Determine the similarity between the image text features and the error correction text features, and define the similarity between the image text features and the error correction text features as the relevance between the translated error correction text and the translation interface image.

[0010] Optionally, in another possible implementation of the first aspect, after confirming that the error correction work order has been approved, the following steps are also included: By using a pre-set work order review model, the system generates modification information corresponding to the error correction work order based on the translation error correction feedback information. The modification information includes the correct translation text corresponding to the translation error correction feedback information. The modified information is sent to the designated handler so that the designated handler can process the error correction work order based on the modified information.

[0011] Secondly, this application also provides a training method for a large-scale work order review model, including: obtaining multiple error correction work orders corresponding to the target application submitted by the user; inputting each error correction work order into an initial large-scale work order review model, and generating model review results corresponding to each error correction work order through the initial large-scale work order review model; obtaining the manual review results corresponding to each error correction work order; and iteratively training the initial large-scale work order review model based on the difference between the model review results and the manual review results corresponding to each error correction work order to generate a preset large-scale work order review model.

[0012] Thirdly, this application also provides a work order review device, comprising: a first acquisition module for acquiring a correction work order corresponding to a target application submitted by a user; a first determination module for determining the translation correction feedback information corresponding to the correction work order based on the correction work order; a second determination module for determining whether the translation correction feedback information meets preset language standardization conditions and preset relevance conditions through a preset work order review model; and a third determination module for determining that the correction work order review is passed if the translation correction feedback information meets the preset language standardization conditions and preset relevance conditions.

[0013] In one possible implementation of the third aspect, the aforementioned translation error correction feedback information includes translation error correction text and translation interface images; the aforementioned preset language standardization conditions include at least one blacklist field; and the aforementioned preset relevance conditions include a relevance threshold. Correspondingly, the aforementioned second determining module includes: The input unit is used to input translation error correction feedback information into the preset work order review model; The first determining unit is used to determine whether the translated error correction text includes blacklist fields through a preset work order review model; The second determining unit is used to determine whether the translation error correction feedback information meets the preset language specification conditions when the translation error correction text does not include any blacklist field. The third determining unit is used to determine the relevance between the translated error correction text and the translation interface image by reviewing the pre-set work order model; The fourth determining unit is used to determine whether the translation error correction feedback information meets the preset relevance condition when the relevance is greater than or equal to the relevance threshold.

[0014] Optionally, in another possible implementation of the third aspect, the aforementioned first determining unit is specifically used for: The translated text undergoes word segmentation to generate a word segmentation sequence, wherein the word segmentation sequence includes at least one word. Each word segment is matched against each blacklist field to determine the matching degree between each word segment and each blacklist field. If any word segment matches any blacklist field with a degree greater than a preset matching threshold, the translated text to be corrected includes the blacklist field. If the matching degree between all word segments and each blacklist field is less than or equal to the preset matching degree threshold, it is determined that the translated and corrected text does not include any blacklist field.

[0015] Optionally, in another possible implementation of the third aspect, the aforementioned third determining unit is specifically used for: Extract text from images on the translation interface to generate image text; Feature extraction is performed on the image text to determine the image text features corresponding to the image text; Feature extraction is performed on the translation correction text to determine the features of the correction text corresponding to the translation correction text; Determine the similarity between the image text features and the error correction text features, and define the similarity between the image text features and the error correction text features as the relevance between the translated error correction text and the translation interface image.

[0016] Optionally, in another possible implementation of the third aspect, the aforementioned work order approval device further includes: The third generation module is used to generate the modification information corresponding to the error correction work order based on the translation error correction feedback information through the preset work order review model. The modification information includes the correct translation text corresponding to the translation error correction feedback information. The sending module is used to send the modified information to the preset handler so that the preset handler can process the error correction work order according to the modified information.

[0017] Fourthly, this application also provides a training device for a large-scale work order review model, comprising: a second acquisition module for acquiring multiple error correction work orders corresponding to a target application submitted by a user; a first generation module for inputting each error correction work order into an initial large-scale work order review model, and generating model review results corresponding to each error correction work order through the initial large-scale work order review model; a third acquisition module for acquiring the manual review results corresponding to each error correction work order; and a second generation module for iteratively training the initial large-scale work order review model based on the difference between the model review results and the manual review results corresponding to each error correction work order, so as to generate a preset large-scale work order review model.

[0018] Fifthly, this application also provides an electronic device. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the computer program to implement any of the implementations of the first aspect described above.

[0019] Sixthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the method of any of the implementations of the first aspect described above.

[0020] Seventhly, this application also provides a computer program product that, when run on an electronic device, causes the electronic device to execute any of the implementation methods of the first aspect described above.

[0021] The beneficial effects of this application embodiment compared with the prior art are: by using a preset work order review model to perform language standardization and relevance checks on the translation error correction feedback information of the error correction work orders submitted by users, the error correction work orders can be reviewed, thereby improving the review efficiency of error correction work orders. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating a work order review method provided in an embodiment of this application; Figure 2 This is a flowchart illustrating a work order review method provided in another embodiment of this application; Figure 3 This is a flowchart illustrating the training method of a large-scale work order review model provided in one embodiment of this application; Figure 4 This is a flowchart illustrating the training method for a large-scale work order review model provided in another embodiment of this application; Figure 5 This is a schematic diagram of the work order review device provided in one embodiment of this application; Figure 6 This is a schematic diagram of the structure of a training device for a large work order review model provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0024] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0025] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0026] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0027] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0028] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0029] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0030] The following description, with reference to the accompanying drawings, details the work order review method, apparatus, electronic device, storage medium, computer program, and training method for the large-scale work order review model provided in this application.

[0031] Figure 1 A flowchart illustrating a work order review method provided in an embodiment of this application is shown.

[0032] Step 101: Obtain the bug fix ticket corresponding to the target application submitted by the user.

[0033] It should be noted that the work order review method of this application embodiment can be executed by the work order review device of this application embodiment. The work order review device of this application embodiment can be configured in any electronic device to execute the work order review method of this application embodiment.

[0034] In this context, "user" refers to the operator using the target application, which may include ordinary end-users, technical professionals, language experts, or testers. However, it is not limited to these categories.

[0035] The target application can be an application used in different regions, such as a vehicle diagnostic application corresponding to a vehicle diagnostic device, but it is not limited to this.

[0036] In this context, a correction ticket can be a structured digital record used to systematically collect and manage user feedback on translation errors. A correction ticket may include the following information: a unique ticket number, a submission timestamp, a user identifier (which may be anonymized), an image of the current translation interface, the user's suggested correct translation, a reason for the correction, and relevant contextual information such as application module, software version, and device environment, but is not limited to these.

[0037] In one possible implementation of this application, a "Feedback" or "Correction" button can be provided to the user in the target application interface. After the user clicks the button, the system can automatically capture the translation content of the current screen and generate a translation interface image based on the captured content. A feedback form can also pop up, which can include fields such as suggested correct translation, error type, reason for modification, specific location of the error, scope of impact, and urgency of modification. This allows users to provide feedback on translation errors through the feedback form. After the user confirms and submits the feedback, a correction ticket can be generated based on the user's feedback. The correction tickets submitted by the user can be obtained through a preset ticket feedback interface.

[0038] Step 102: Based on the error correction work order, determine the translation error correction feedback information corresponding to the error correction work order.

[0039] The translation error correction feedback information may include translation error correction text, such as the correct translation suggestion corresponding to the translation error pointed out by the user, the error type, the reason for modification, the specific location of the error, the scope of impact, and the urgency of modification, and may also include translation interface images, but is not limited to these.

[0040] In one possible implementation of this application, natural language processing can be used to parse the error correction work order and extract the correct translation suggestions, error type, reason for modification, specific location of the error, scope of impact, and urgency of modification corresponding to the translation error reported by the user in the error correction work order, so as to generate translation error correction feedback information.

[0041] It should be noted that step 102 can filter out information in the error correction work order that is not related to the work order review, such as the unique work order number, submission timestamp, software version, equipment environment, etc., and extract translation error correction feedback information related to the work order review.

[0042] Step 103: Using the pre-set work order review model, determine whether the translation error correction feedback information meets the pre-set language standard conditions and pre-set relevance conditions.

[0043] Among them, the preset work order review model can be a pre-trained analysis model specifically designed for reviewing translation error correction feedback information.

[0044] The preset language standardization conditions can be a set of predefined rules and standards used to judge whether translation error correction feedback information conforms to language and civility norms. For example, preset language standardization conditions could be "not containing any expressions that violate public order and good morals, platform content policies, or laws and regulations." However, they are not limited to this.

[0045] For example, the preset language standard conditions can include a set of prohibited words or phrases such as sensitive words, uncivilized language, and extreme remarks that are forbidden in translation error correction feedback information.

[0046] Among them, the preset relevance conditions can be used to determine whether the translation error feedback information provided by users is related to the actual translation error problem, so as to filter out the error correction translation information that is not related to the translation error problem, such as emotional venting, advertising, irrelevant topics or meaningless characters.

[0047] As one possible approach, translation error correction feedback, preset language standardization conditions, and preset relevance conditions can be input into a preset work order review model. The preset work order review model can then analyze the translation error correction feedback to determine whether it meets the preset language standardization conditions and preset relevance conditions.

[0048] Optionally, the above translation error correction feedback information includes the translation error correction text and the translation interface image; the above preset language standardization conditions include at least one blacklist field; the above preset relevance conditions include a relevance threshold; and step 103 may include: Input the translation error correction feedback information into the preset work order review model; By using a pre-set work order review model, it can be determined whether the translated error-correction text includes blacklist fields. If the translation correction text does not include any blacklist field, it is determined that the translation correction feedback information meets the preset language specification conditions; By using a pre-set work order review model, the relevance between the translated error correction text and the translation interface images is determined. If the relevance is greater than or equal to the relevance threshold, the translation error correction feedback information is determined to meet the preset relevance condition.

[0049] As one possible implementation, the preset language standard conditions include at least one blacklist field. These blacklist fields can be determined based on social norms, platform content policies, or laws and regulations. The blacklist fields can include various sensitive words, uncivilized language, extreme statements, and other prohibited words or phrases. However, they are not limited to this.

[0050] First, the system can obtain various preset blacklist fields and input the translation correction feedback information and each blacklist field into the preset work order review model. The preset work order review model can then iterate through each blacklist field in the preset language specification conditions. For each blacklist field, a string matching operation can be performed to determine whether the blacklist field appears completely in the user-submitted translation correction text. If the translation correction text includes any blacklist field, it can be determined that the translation correction feedback information does not meet the preset language specification conditions. If, after iterating through all blacklist fields, the translation correction text does not include any blacklist field, it can be determined that the translation correction feedback information meets the preset language specification conditions.

[0051] For example, if the blacklist field contains the "inferior product" field, and the translation correction text is "This translation is as bad as an inferior product", then the preset work order review big data model can be used to search for the "inferior product" field in the translation correction text. Since the field exists in the translation correction text, the preset work order review big data model can be used to determine that the translation correction feedback information does not meet the preset language specification conditions.

[0052] It should be noted that the blacklist fields and translation correction texts listed above are merely examples. In actual use, the blacklist fields and translation correction texts can be determined according to actual usage requirements and application scenarios. This application embodiment does not limit this.

[0053] Optionally, to achieve more comprehensive and flexible matching, the translated error correction text and the blacklist field can be converted to lowercase before string matching to eliminate missed detections caused by case differences. For certain specific scenarios, some punctuation marks or spaces in the translated error correction text can also be ignored.

[0054] Optionally, to prevent mismatches, each word segment of the translated and corrected text can be extracted first, and each word segment can be matched with each blacklist field to improve the accuracy of the matching, thereby improving the accuracy of the work order review. That is, in one possible implementation of this application embodiment, determining whether the translated and corrected text includes a blacklist field may include: The translated text undergoes word segmentation to generate a word segmentation sequence, wherein the word segmentation sequence includes at least one word. Each word segment is matched against each blacklist field to determine the matching degree between each word segment and each blacklist field. If any word segment matches any blacklist field with a degree greater than a preset matching threshold, the translated text to be corrected includes the blacklist field. If the matching degree between all word segments and each blacklist field is less than or equal to the preset matching degree threshold, it is determined that the translated and corrected text does not include any blacklist field.

[0055] In one possible implementation, the translated and corrected text can be segmented. For example, a specialized dictionary can be pre-built for each supported language, containing commonly used vocabulary, terminology, and domain-specific terms. A maximum forward matching algorithm can be used to scan the translated and corrected text from left to right, matching the longest substring against the dictionary at each step. For words not included in the dictionary, statistical methods can be used to assist in identification, thereby generating a segmented sequence containing at least one word.

[0056] For example, if the correct translation of the text is "The correct translation of this sentence is engine oil pressure too low alarm", then the word segmentation corresponding to the generated text can be "this sentence", "of", "correct translation", "for", "engine", "oil pressure", "too low", and "alarm".

[0057] Next, each word segment can be matched against its respective blacklist field, and the matching degree between each word segment and each blacklist field can be determined. Based on the matching degree of each word segment against its respective blacklist field, it can be determined whether the translated and corrected text includes any blacklist field, thus determining whether the translated and corrected text meets the preset language standardization conditions. For example, if the matching degree between any word segment and any blacklist field is greater than the preset matching degree threshold, it can be determined that the translated and corrected text includes any blacklist field, meaning the translated and corrected text does not meet the preset language standardization conditions. If the matching degree between all word segments and each blacklist field is less than or equal to the preset matching degree threshold, it can be determined that the translated and corrected text does not include any blacklist field, meaning the translated and corrected text meets the preset language standardization conditions.

[0058] As an example, each word segment can be precisely matched against each blacklist field, with a preset matching threshold of 0. If a match is successful, meaning a blacklist field is exactly the same as a certain word segment, then the matching degree corresponding to that word is determined to be 1, and it can be determined that the translated and corrected text includes that blacklist field.

[0059] As another example, each word segment can be mapped to a word segment vector, and each blacklist field can be mapped to a blacklist vector. Then, the cosine distance between each word segment vector and each blacklist vector can be calculated, and the cosine distance between each word segment vector and each blacklist vector can be used as the matching degree between each word segment and each blacklist field.

[0060] Next, after determining whether the translation correction feedback information meets the preset language standardization conditions, the content of the translation correction text can be parsed through the preset work order review model. The content of the translation interface image can also be extracted. The parsed translation correction text content is compared with the translation interface image content to determine the relevance between the translation correction text and the translation interface image. If the relevance is greater than or equal to the relevance threshold, the translation correction feedback information is determined to meet the preset relevance conditions. If the relevance is less than the relevance threshold, the translation correction feedback information is determined to not meet the preset relevance conditions.

[0061] As an example, determining the relevance between the translation correction text and the translation interface image can include: Extract text from images on the translation interface to generate image text; Feature extraction is performed on the image text to determine the image text features corresponding to the image text; Feature extraction is performed on the translation correction text to determine the features of the correction text corresponding to the translation correction text; Determine the similarity between the image text features and the error correction text features, and define the similarity between the image text features and the error correction text features as the relevance between the translated error correction text and the translation interface image.

[0062] Among them, image text can be the text included in the image of the translation interface.

[0063] One possible approach is to extract text from translation interface images using optical character recognition (OCR) technology. This can include preprocessing the translation interface image (such as binarization and noise reduction), image text region localization, character segmentation, and recognition to generate image text. Then, natural language processing (NLP) techniques can be used to extract features from both the image text and the translation correction text. For example, a pre-trained large language model can be used to generate image text features corresponding to the image text and error correction text features corresponding to the translation correction text.

[0064] It should be noted that the methods for feature extraction from image text and feature extraction from translation error correction text should be the same or compatible to ensure that the extracted features of both are located in the same vector space, thus making them comparable.

[0065] Next, the similarity between the image text features and the error correction text features can be determined. The similarity can be calculated using measurement methods such as cosine similarity, Euclidean distance, or Manhattan distance. The similarity between the image text features and the error correction text features can be defined as the relevance between the translated error correction text and the translated interface image.

[0066] Step 104: If the translation error correction feedback information meets the preset language standard conditions and preset relevance conditions, the error correction work order is approved.

[0067] In one possible implementation of this application, when the translation error correction feedback information meets the preset language standard conditions and the translation error correction feedback information meets the preset relevance conditions, the error correction work order can be determined to have passed the review, and the review result of the error correction work order can be output as "review passed" through the preset work order review model. When the translation error correction feedback information does not meet the preset language standard conditions or the translation error correction feedback information does not meet the preset relevance conditions, the error correction work order can be determined to have failed the review, and the review result of the error correction work order can be output as "failed to pass" through the preset work order review model.

[0068] Optionally, when a correction work order fails to pass the review, the corresponding reason for the failure can be generated, such as "not meeting the preset language specification", "not meeting the preset relevance condition", or "not meeting both the preset language specification and the preset relevance condition".

[0069] As an example, a pre-defined work order review model can be used to evaluate translation correction feedback. If the corrected text in the feedback does not contain any blacklist fields, and the relevance between the corrected text and the translation interface image is greater than or equal to a relevance threshold, the work order's review result is "Approved." If the corrected text in the feedback contains any blacklist fields, or the relevance between the corrected text and the translation interface image is less than a relevance threshold, the work order's review result is "Not Approved."

[0070] Optionally, the review results of each error correction work order by the preset work order review model can be stored in the database for subsequent manual auditing, or used as training data to continuously optimize the preset work order review model.

[0071] Optionally, after confirming that the error correction work order has been approved, the following steps are also included: By using a pre-set work order review model, the system generates modification information corresponding to the error correction work order based on the translation error correction feedback information. The modification information includes the correct translation text corresponding to the translation error correction feedback information. The modified information is sent to the designated handler so that the designated handler can process the error correction work order based on the modified information.

[0072] In one possible implementation, a pre-defined work order review model can be used to analyze translation error correction feedback information. Based on the user-suggested correct translation, translation interface images, and explanations of the reasons for modification in the feedback information, a correct translation text can be generated. This, combined with other information included in the error correction work order, such as application module, software version, and device environment, will generate the corresponding modification information for the work order.

[0073] Next, the modification information can be sent to a pre-defined handler, who can then process the error correction ticket based on the modification information. For example, the pre-defined handler can publish the corrected translation to the original location of the translation error in the error correction ticket based on the modification information.

[0074] In one possible implementation of this application, the target application is a vehicle diagnostic application, and the work order review method provided in one embodiment of this application can be as follows: Figure 2The flowchart shown illustrates the process. First, it retrieves error correction work orders submitted by users through the vehicle diagnostic application. Second, it extracts translation error correction feedback information from these work orders and inputs it into a preset work order review model. This model determines whether the feedback information meets preset language standardization and relevance conditions. If both conditions are met, the work order is approved. Modification information is then generated based on the feedback and other information in the work order and sent to a preset handler to correct translation errors. If neither condition is met, the work order fails the review. Next, the model review results for each work order are sent to the database, and the work order itself is sent to a preset reviewer for manual review. The manual review results are retrieved and stored in the database. Finally, the preset work order review model is optimized based on the model and manual review results for each work order in the database.

[0075] The work order review method provided in this application embodiment uses a preset work order review model to perform language standardization and relevance checks on the translation and error correction feedback information of the error correction work orders submitted by users, thereby improving the efficiency of error correction work order review.

[0076] Figure 3 A flowchart illustrating a work order review method provided in an embodiment of this application is shown.

[0077] Step 301: Obtain multiple bug fixing tickets corresponding to the target application submitted by the user.

[0078] The specific implementation of step 301 is the same as that of step 101 above, and will not be repeated here in the embodiments of this application.

[0079] Step 302: Input each error correction work order into the initial work order review model, and generate the model review results corresponding to each error correction work order through the initial work order review model.

[0080] The initial work order review model can be pre-built based on machine learning techniques, such as deep learning frameworks, and may include modules such as image text extraction, text feature extraction, and text matching.

[0081] The specific implementation method of step 302 is the same as that of steps 102 to 104 above, which determines the error correction work order review result by using a preset work order review big model. The embodiments of this application will not be described again here.

[0082] Step 303: Obtain the manual review results corresponding to each error correction work order.

[0083] In one possible implementation of this application, each error correction work order can be sent to a preset work order reviewer, enabling the preset work order reviewer to review each error correction work order and generate a manual review result for each error correction work order. The manual review result of each error correction work order by the preset work order reviewer can be obtained through a preset interface and stored in a database.

[0084] Step 304: Based on the difference between the model review result and the manual review result corresponding to each error correction work order, iteratively train the initial work order review model to generate a preset work order review model.

[0085] In one possible implementation of this application, the model review result and the human review result corresponding to each error correction work order can be input into a pre-trained scoring model. The scoring model can analyze the difference between the model review result and the human review result corresponding to each error correction work order from four dimensions: the model review result is approved and the human review result is also approved; the model review result is not approved but the human review result is approved; the model review result is not approved and the human review result is also not approved; and the model review result is approved but the human review result is not approved.

[0086] A scoring model can be used to generate the review confidence score of the initial work order review model based on the difference between the model review result and the human review result for each error correction work order. For example, the more discrepancies between the model review result and the human review result, the lower the review confidence score. If the review confidence score of the initial work order review model is greater than or equal to the confidence score threshold, the initial work order review model can be identified as the target work order review model. If the review confidence score of the initial work order review model is less than the confidence score threshold, each error correction work order can be used as a training sample, and the corresponding human review result for each error correction work order can be used as the labeled data for each training sample to generate a training dataset. The initial work order review model can then be iteratively trained and updated using the training dataset until the review confidence score of the updated initial work order review model is greater than or equal to the confidence score threshold. The updated initial work order review model can then be identified as the target work order review model.

[0087] In one possible implementation of this application, the target application is a vehicle diagnostic application, and the training method for the large-scale work order review model provided in one embodiment of this application can be as follows: Figure 4The flowchart shown is executed as follows. First, multiple error correction work orders submitted by the user through the vehicle diagnostic application can be obtained. Second, each error correction work order can be reviewed by a preset work order review model and preset reviewers. The model review results of each error correction work order generated by the initial work order review model and the manual review results of each error correction work order submitted by the preset reviewers can be input into the scoring model. The confidence level of the initial work order review model is determined by the scoring model. The model review results of each error correction work order generated by the initial work order review model and the manual review results of each error correction work order submitted by the preset reviewers can be stored in the database. The initial work order review model can be iteratively trained using the model review results and manual review results corresponding to each error correction work order in the database, as well as the confidence level generated by the scoring model, until the confidence level of the trained initial work order review model reaches the confidence level threshold.

[0088] In one possible implementation, the target work order review model can be used as the preset work order review model in the above embodiments, and the error correction work orders can be reviewed as described in the above embodiments through the target work order review model.

[0089] The training method for the large-scale work order review model provided in this application embodiment iteratively trains the initial large-scale work order review model based on the model review results and manual review results corresponding to each error correction work order to generate a target large-scale work order review model. Thus, the error correction work orders can be reviewed through the target large-scale work order review model, thereby improving the review efficiency of error correction work orders.

[0090] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0091] Corresponding to the work order review method described in the above embodiments, Figure 5 A structural block diagram of the work order review device provided in the embodiments of this application is shown. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0092] Reference Figure 5 The device 50 includes: The first acquisition module 51 is used to acquire the error correction work order corresponding to the target application submitted by the user. The first determining module 52 is used to determine the translation error correction feedback information corresponding to the error correction work order based on the error correction work order; The second determination module 53 is used to determine whether the translation error correction feedback information meets the preset language standard conditions and preset relevance conditions through the preset work order review big model; The third determination module 54 is used to determine whether the error correction work order has been approved if the translation error correction feedback information meets the preset language standard conditions and preset relevance conditions.

[0093] In practical use, the work order review device provided in this application embodiment can be configured in any electronic device to execute the aforementioned work order review method.

[0094] The work order review device provided in this application embodiment performs language standardization and relevance verification on the translation and error correction feedback information of the error correction work orders submitted by users through a preset work order review model, thereby improving the review efficiency of error correction work orders.

[0095] In one possible implementation of this application, the translation error correction feedback information includes the translation error correction text and the translation interface image; the preset language standardization conditions include at least one blacklist field; and the preset relevance conditions include a relevance threshold. Correspondingly, the second determining module 53 includes: The input unit is used to input translation error correction feedback information into the preset work order review model; The first determining unit is used to determine whether the translated error correction text includes blacklist fields through a preset work order review model; The second determining unit is used to determine whether the translation error correction feedback information meets the preset language specification conditions when the translation error correction text does not include any blacklist field. The third determining unit is used to determine the relevance between the translated error correction text and the translation interface image by reviewing the pre-set work order model; The fourth determining unit is used to determine whether the translation error correction feedback information meets the preset relevance condition when the relevance is greater than or equal to the relevance threshold.

[0096] Optionally, in another possible implementation of this application, the first determining unit is specifically used for: The translated text undergoes word segmentation to generate a word segmentation sequence, wherein the word segmentation sequence includes at least one word. Each word segment is matched against each blacklist field to determine the matching degree between each word segment and each blacklist field. If any word segment matches any blacklist field with a degree greater than a preset matching threshold, the translated text to be corrected includes the blacklist field. If the matching degree between all word segments and each blacklist field is less than or equal to the preset matching degree threshold, it is determined that the translated and corrected text does not include any blacklist field.

[0097] Optionally, in another possible implementation of this application, the aforementioned third determining unit is specifically used for: Extract text from images on the translation interface to generate image text; Feature extraction is performed on the image text to determine the image text features corresponding to the image text; Feature extraction is performed on the translation correction text to determine the features of the correction text corresponding to the translation correction text; Determine the similarity between the image text features and the error correction text features, and define the similarity between the image text features and the error correction text features as the relevance between the translated error correction text and the translation interface image.

[0098] Optionally, in another possible implementation of this application, the above-mentioned work order review device 50 further includes: The third generation module is used to generate the modification information corresponding to the error correction work order based on the translation error correction feedback information through the preset work order review model. The modification information includes the correct translation text corresponding to the translation error correction feedback information. The sending module is used to send the modified information to the preset handler so that the preset handler can process the error correction work order according to the modified information.

[0099] Corresponding to the training method of the large-scale work order review model described in the above embodiment, Figure 6 The diagram shows a structural block diagram of the training device for the large-scale work order review model provided in the embodiments of this application. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0100] Reference Figure 6 The device 60 includes: The second acquisition module 61 is used to acquire multiple error correction work orders corresponding to the target application submitted by the user. The first generation module 62 is used to input each error correction work order into the initial work order review model, and generate the model review results corresponding to each error correction work order through the initial work order review model. The third acquisition module 63 is used to acquire the manual review results corresponding to each error correction work order; The second generation module 64 is used to iteratively train the initial work order review model based on the difference between the model review result and the manual review result corresponding to each error correction work order, so as to generate a preset work order review model.

[0101] The training device for the work order review model provided in this application embodiment iteratively trains the initial work order review model based on the model review results and manual review results corresponding to each error correction work order to generate a target work order review model. This allows the error correction work orders to be reviewed through the target work order review model, thereby improving the review efficiency of error correction work orders.

[0102] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0103] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0104] To implement the above embodiments, this application also proposes an electronic device.

[0105] Figure 7 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application.

[0106] like Figure 7 As shown, the above-mentioned electronic device 200 includes: The system includes a memory 210 and at least one processor 220, and a bus 230 connecting different components (including the memory 210 and the processor 220). The memory 210 stores a computer program, and when the processor 220 executes the program, it implements the work order review method of the present application embodiment.

[0107] Bus 230 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0108] Electronic device 200 typically includes a variety of electronic device readable media. These media can be any available media that can be accessed by electronic device 200, including volatile and non-volatile media, removable and non-removable media.

[0109] Memory 210 may also include computer system readable media in the form of volatile memory, such as random access memory (RAM) 240 and / or cache memory 250. Electronic device 200 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 260 may be used to read and write non-removable, non-volatile magnetic media (… Figure 7 Not shown; usually referred to as a "hard drive"). Although Figure 7 As not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 230 via one or more data media interfaces. Memory 210 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this application.

[0110] A program / utility 280 having a set (at least one) of program modules 270 may be stored in, for example, memory 210. Such program modules 270 include—but are not limited to—an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 270 typically perform the functions and / or methods described in the embodiments of this application.

[0111] Electronic device 200 can also communicate with one or more external devices 290 (e.g., keyboard, pointing device, display 291, etc.), and with one or more devices that enable a user to interact with electronic device 200, and / or with any device that enables electronic device 200 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 292. Furthermore, electronic device 200 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 293. As shown, network adapter 293 communicates with other modules of electronic device 200 via bus 230. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0112] The processor 220 performs various functional applications and data processing by running programs stored in the memory 210.

[0113] It should be noted that the implementation process and technical principles of the electronic device in this embodiment are explained in the foregoing description of the work order review method in this application embodiment, and will not be repeated here.

[0114] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps described in the above-described method embodiments.

[0115] This application provides a computer program product that, when run on an electronic device, enables the electronic device to perform the steps described in the various method embodiments above.

[0116] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographic device / electronic device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0117] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0118] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0119] In the embodiments provided in this application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0120] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0121] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A work order review method, characterized in that, include: Retrieve the bug fix ticket corresponding to the target application submitted by the user; Based on the error correction work order, determine the translation error correction feedback information corresponding to the error correction work order; The translation error correction feedback information is determined by using a pre-set work order review model to determine whether it meets the pre-set language standard conditions and pre-set relevance conditions. If the translation error correction feedback information meets the preset language standard conditions and the preset relevance conditions, the error correction work order is determined to be approved.

2. The method as described in claim 1, characterized in that, The translation error correction feedback information includes the translation error correction text and the translation interface image. The preset language standardization conditions include at least one blacklist field. The preset relevance conditions include a relevance threshold. The step of determining whether the translation error correction feedback information meets the preset language standardization conditions and preset relevance conditions through a preset work order review model includes: The translation error correction feedback information is input into the preset work order review model; The preset work order review model is used to determine whether the translated error correction text includes the blacklist field. If the translation correction text does not include any of the blacklist fields, it is determined that the translation correction feedback information meets the preset language specification conditions; The correlation between the translated error correction text and the translation interface image is determined using the preset work order review model. If the relevance is greater than or equal to the relevance threshold, the translation error correction feedback information is determined to meet the preset relevance condition.

3. The method as described in claim 2, characterized in that, Determining whether the translated error-corrected text includes the blacklist field includes: The translated error-correction text is segmented to generate a segmentation sequence, wherein the segmentation sequence includes at least one segment; Each of the word segments is matched with each of the blacklist fields to determine the matching degree between each of the word segments and each of the blacklist fields. If the matching degree between any of the word segments and any of the blacklist fields is greater than a preset matching degree threshold, it is determined that the translation correction text includes the blacklist field; If the matching degree between all the word segments and each of the blacklist fields is less than or equal to the preset matching degree threshold, it is determined that the translated error-correcting text does not include any of the blacklist fields.

4. The method as described in claim 2, characterized in that, Determining the relevance between the translated error-correction text and the translation interface image includes: Text is extracted from the translation interface image to generate image text; Feature extraction is performed on the image text to determine the image text features corresponding to the image text; Feature extraction is performed on the translated error-correcting text to determine the error-correcting text features corresponding to the translated error-correcting text; The similarity between the image text features and the error correction text features is determined, and the similarity between the image text features and the error correction text features is determined as the relevance between the translation error correction text and the translation interface image.

5. The method according to any one of claims 1-4, characterized in that, After confirming that the error correction work order has been approved, the process also includes: Based on the preset work order review model, the modification information corresponding to the error correction work order is generated according to the translation error correction feedback information. The modification information includes the correct translation text corresponding to the translation error correction feedback information. The modified information is sent to a pre-defined handler so that the pre-defined handler can process the error correction work order based on the modified information.

6. A training method for a large-scale work order review model, characterized in that, include: Retrieve multiple bug fix tickets corresponding to the target application submitted by the user; Each of the error correction work orders is input into the initial work order review model, and the model review result corresponding to each of the error correction work orders is generated through the initial work order review model. Obtain the manual review results corresponding to each of the aforementioned error correction work orders; Based on the difference between the model review result and the manual review result corresponding to each error correction work order, the initial work order review model is iteratively trained to generate a preset work order review model.

7. A work order verification device, characterized in that, include: The first acquisition module is used to acquire the error correction work order corresponding to the target application submitted by the user. The first determining module is used to determine the translation error correction feedback information corresponding to the error correction work order based on the error correction work order; The second determining module is used to determine whether the translation error correction feedback information meets the preset language standard conditions and preset relevance conditions through a preset work order review model. The third determining module is used to determine that the error correction work order has been approved if the translation error correction feedback information meets the preset language standard conditions and the preset relevance conditions.

8. A training device for a large-scale work order review model, characterized in that, include: The second acquisition module is used to acquire multiple error correction work orders corresponding to the target application submitted by the user. The first generation module is used to input each of the error correction work orders into the initial work order review model, and generate the model review result corresponding to each of the error correction work orders through the initial work order review model. The third acquisition module is used to acquire the manual review results corresponding to each of the error correction work orders; The second generation module is used to iteratively train the initial work order review model based on the difference between the model review result and the manual review result corresponding to each error correction work order, so as to generate a preset work order review model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it causes the electronic device to implement the method as described in any one of claims 1 to 6.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by an electronic device, it implements the method as described in any one of claims 1 to 6.

Images