Defect information processing method and device and storage medium

Abstract

The invention relates to a defect information processing method and device and a storage medium. The defect information processing method comprises the steps that a first model is called, the first model is used for conducting defect management on all sub-processes in process management, and the process management comprises one or more sub-processes; obtaining defect information in process management based on the first model, and positioning a defect problem based on the defect information; and in response to any sub-process, positioning a defect problem based on the defect information, and generating a solution corresponding to the defect problem. And the defect information solving efficiency and the defect information solving accuracy are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a defect information processing method, device and storage medium. Background Art

[0002] With the continuous development of computer technology, various technical means are used in the software development process to comprehensively assist and optimize all aspects of defect management, including defect discovery, recording, tracking, analysis, resolution and verification.

[0003] Among the related technologies, artificial intelligence (AI) models based on large-scale parameters (over 10 billion levels) have demonstrated strong semantic understanding, text generation and code review capabilities. However, the related systems are insufficient in full-process assistance and are unable to fully apply their assistance capabilities to software defect management, failing to effectively leverage the advantages of artificial intelligence to enhance the automation capabilities of defect management. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides a defect information processing method, device and storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, a defect information processing method is provided, comprising calling a first model, wherein the first model is used to perform defect management on each sub-process in a process management, wherein the process management includes one or more sub-processes; obtaining defect information in the process management based on the first model, and locating defect problems based on the defect information; in response to locating defect problems based on the defect information for any sub-process, generating a solution corresponding to the defect problem.

[0006] In one embodiment, the method of locating the defect problem based on the defect information includes: semantically encoding the defect information to obtain semantic text, and performing content retrieval on the semantic text based on a content index library to obtain a content block identifier and a content block text, wherein the content index library is used to store an index relationship between a content block identifier and a content block text; based on the content block identifier, determining a semantically encoded text corresponding to the content block identifier in a semantic encoding library; semantically sorting the semantic text according to a similarity between the content block text and the semantically encoded text; and locating the defect problem based on the semantic text after the semantic sorting.

[0007] In one embodiment, the defect information includes multimodal data, and generating a solution corresponding to the defect problem includes: confirming the information model of the defect information and performing self-supervised training according to the information type, the defect information model includes at least one of a visual model and a language model; based on the unimodal data obtained by the self-supervised training, obtaining a visual model and a language model that match the unimodal data; aligning the visual features output by the visual model with the text features output by the language model to obtain a solution.

[0008] In one embodiment, the aligning of the visual features output by the visual model and the text features output by the language model to obtain a solution includes: aligning the visual features output by the visual model and the text features output by the language model to obtain a first alignment result; aligning the first alignment result with expected data to obtain a solution.

[0009] In one embodiment, obtaining the expected data includes: calling a second model, where the second model is trained with vertical domain data determined by user privatization requirements; and obtaining the expected data based on the second model.

[0010] According to a second aspect of an embodiment of the present disclosure, a defect information processing device is provided, including: a calling unit, used to execute the call of a first model, wherein the first model is used to perform defect management on each sub-process in a process management, wherein the process management includes one or more sub-processes; a processing unit, used to execute the acquisition of defect information in the process management based on the first model, and locate the defect problem based on the defect information; and a generating unit, used to generate a solution corresponding to the defect problem in response to locating the defect problem based on the defect information for any sub-process.

[0011] In one embodiment, the processing unit locates the defect problem based on the defect information in the following manner: semantically encodes the defect information to obtain semantic text, and based on a content index library, performs content retrieval on the semantic text to obtain a content block identifier and a content block text, wherein the content index library is used to store an index relationship between a content block identifier and a content block text; based on the content block identifier, determines a semantically encoded text corresponding to the content block identifier in a semantic encoding library; semantically sorts the semantic text according to a similarity between the content block text and the semantically encoded text; and locates the defect problem based on the semantic text after the semantic sorting.

[0012] In one embodiment, the generation unit generates a solution corresponding to the defect problem in the following manner: confirming the information model of the defect information, and performing self-supervised training according to the information type, the defect information model includes at least one of a visual model and a language model; based on the unimodal data obtained by the self-supervised training, obtaining a visual model and a language model that match the unimodal data; aligning the visual features output by the visual model and the text features output by the language model to obtain a solution.

[0013] In one embodiment, the generation unit aligns the visual features output by the visual model with the text features output by the language model to obtain a solution in the following manner: aligning the visual features output by the visual model with the text features output by the language model to obtain a first alignment result; aligning the first alignment result with expected data to obtain a solution.

[0014] In one embodiment, the generation unit obtains the expected data in the following manner: calling a second model, where the second model is trained with vertical domain data determined by user privatization requirements; and obtaining the expected data based on the second model.

[0015] According to a third aspect of an embodiment of the present disclosure, a terminal device is provided, comprising: a processor; and a memory for storing processor executable instructions; wherein the processor is configured to execute the executable instructions to execute the display control method in the first aspect or any one of the implementations of the first aspect.

[0016] According to the fourth aspect of an embodiment of the present disclosure, a storage medium is provided, in which instructions are stored. When the instructions in the storage medium are executed by a processor of a terminal, the terminal device is enabled to execute the defect information processing method in the first aspect or any one of the embodiments of the first aspect.

[0017] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: based on the defect problem obtained by the first model and the defect problem located according to the defect information, the first model finally generates a solution to the problem, thereby improving the automation capability of defect management and the accuracy of the problem solution.

[0018] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0020] Figure 1The figure is a flowchart of a defect information processing method according to an exemplary embodiment.

[0021] Figure 2 The invention is a flow chart showing a method for locating a defect problem according to an exemplary embodiment.

[0022] Figure 3 The invention is a flow chart showing a method for generating a solution to a defect problem according to an exemplary embodiment.

[0023] Figure 4 The figure is a flow chart showing a method of obtaining a solution according to an exemplary embodiment.

[0024] Figure 5 The invention is a flowchart of obtaining expected data according to an exemplary embodiment.

[0025] Figure 6 It is a schematic diagram of a full-process assisted defect management system based on a multimodal AI large model.

[0026] Figure 7 It is a schematic diagram of the pre-training stage of a semantic encoding module.

[0027] Figure 8 It is a schematic diagram of the fine-tuning stage of a semantic encoding module.

[0028] Fig. 9 It is a schematic diagram of the pre-training stage of a large multi-modal AI model with large-scale parameters.

[0029] Fig.10 It is a schematic diagram of the fine-tuning stage of a large multi-modal AI model with large parameters.

[0030] Fig.11 It is a schematic diagram of a large multimodal AI model that uses large-scale parameters.

[0031] Fig.12 The figure is a block diagram of a defect information processing device according to an exemplary embodiment.

[0032] Fig.13 The invention is a block diagram of a device for defect information processing according to an exemplary embodiment. DETAILED DESCRIPTION

[0033] Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure.

[0034] The solution acquisition method provided in the embodiment of the present disclosure is applied to computer application scenarios. The solution acquisition method involved in the embodiment of the present disclosure is mainly applied to software development scenarios.

[0035] In the related technologies, artificial intelligence models based on large-scale parameters are used in software defect management. In the entire defect management chain, from creating defects, screening defects, locating problems, to solving problems, multiple links require human participation, and cannot assist defect management in the entire process. Traditional defect management systems use machine learning methods based on rules or partial modules, and have limited intelligent capabilities. They cannot effectively improve defect management efficiency, and the error rate when creating the management system based on rules is high, and users create a large number of duplicate defects.

[0036] In addition, traditional defect management systems do not have the ability to generate solutions, and R&D personnel are required to locate the root cause of the problem based on user feedback and personal experience.

[0037] In view of this, the present invention provides a software defect management method, which realizes full-process assisted defect management based on a multimodal big model, gives full play to the information extraction, semantic retrieval, intelligent question and answer, knowledge acquisition and code generation capabilities of the multimodal artificial intelligence big model with large-scale parameters, and intelligently assists project developers in the whole process to create defects, screen defects, locate problems, and solve problems efficiently, and realizes an intelligent defect management system with human-computer interaction as the core. It only requires users to feedback problems and tasks to be solved and submit them to the system. The system understands the user's intentions and automatically completes the creation and screening of defects, realizes private defect management and decision support for vertical fields, makes full use of private knowledge data in vertical fields, and constructs a content index library and semantic coding library for massive data, which can quickly retrieve semantically related content according to user needs, and the multimodal big model generates decision suggestions based on the related content.

[0038] Figure 1 FIG. 1 is a flow chart of a defect information processing method according to an exemplary embodiment. Figure 1 As shown, the defect information method is used in a terminal, comprising the following steps:

[0039] In step S11 , the first model is called.

[0040] In the disclosed embodiment, the first model is used to perform defect management on each sub-process in the process management, and the process management includes one or more sub-processes.

[0041] Among them, the first model can be understood as a multimodal data processing model that predicts the output text sequence based on the obtained multimodal information.

[0042] In step S12, defect information in process management is acquired based on the first model, and defect problems are located based on the defect information.

[0043] In the disclosed embodiment, defect information is obtained based on manual input by the user into the first model, or the first model automatically detects defect information in process management, and matches defect problems based on the defect information in the first model.

[0044] Among them, defect information can be understood as different types of information problems that occur in different modules or different links of different modules in process management, and defect problems can be understood as defect information with clear problems obtained after information extraction based on defect information.

[0045] In step S13, in response to locating a defect problem based on defect information for any sub-process, a solution to the corresponding defect problem is generated.

[0046] In the disclosed embodiment, a solution to the defect problem is generated according to the defect problem obtained by matching the defect information in the sub-process.

[0047] In the disclosed embodiment, the first model is used to match the obtained defect information with the defect problem to obtain a more accurate defect problem for the defect information, and then a corresponding solution is generated, thereby improving the accuracy of the solution to the defect problem. In addition, the solution to the defect problem is automatically generated according to the first model, thereby improving the management efficiency of process management.

[0048] Figure 2 FIG. 1 is a flow chart showing a method for locating a defect problem according to an exemplary embodiment. Figure 2 As shown, the following steps are included:

[0049] In step S21, semantic coding is performed on the defect information to obtain semantic text, and content retrieval is performed on the semantic text based on the content index library to obtain a content block identifier and a content block text.

[0050] In the disclosed embodiment, an identifier is established based on the acquired defect information, such as semantic coding, to obtain semantic text, and content retrieval is performed on the semantically coded semantic text based on a content index library to obtain a content block identifier and a content block text.

[0051] Among them, the content index library is used to store content block identifiers and index relationships between content block texts. Semantic coding can be understood as being mainly used to directly call the trained semantic coding module to vectorize the segmented text blocks, so that the encoded vectors have strong semantic relevance.

[0052] The content index library can be understood as directly performing an inverted index of keywords and content block identifiers on the segmented content blocks, so that given keywords can quickly retrieve the corresponding content block identifier and the corresponding content block text. Content retrieval can be understood as content extraction.

[0053] In step S22, based on the content block identifier, a semantically coded text corresponding to the content block identifier is determined in a semantic encoding library.

[0054] In the embodiment of the present disclosure, the semantically coded text is determined according to the obtained content block identifier, for example, the semantically coded text corresponding to the content block identifier is determined in a semantic coding library.

[0055] The semantic coding library can be understood as mapping and storing the content block identifier and the semantic coding library, so as to realize that a given content identifier can be quickly extracted to the corresponding semantic coding vector.

[0056] In step S23, the semantic text is semantically sorted according to the similarity between the content block text and the semantically coded text.

[0057] In the disclosed embodiment, the acquired semantic text is semantically sorted, for example, according to the mapping relationship between the content block text and the semantically coded text, for example, according to the similarity between the content block text and the semantically coded text, the semantic text is semantically sorted.

[0058] The similarity can be understood as that, in the process of semantic encoding, the semantic text is vector encoded, and the vector encoding has a strong semantic correlation, that is, the encoding vectors of semantically related text contents have a high similarity.

[0059] In step S24, the defect problem is located based on the semantically sorted semantic text.

[0060] In the disclosed embodiment, defect problems are located based on semantic texts with higher relevance after semantic sorting.

[0061] In the disclosed embodiment, semantic text is obtained by encoding defect information, and the similarity between the content block text and the semantically encoded text is determined based on the semantic text. The semantic text with high similarity is determined, and then the defect problem is determined, thereby improving the accuracy of the defect information and facilitating the subsequent acquisition of solutions based on the accurate defect information.

[0062] Figure 3 FIG. 1 is a flow chart showing a method of generating a solution to a defect problem according to an exemplary embodiment. Figure 3 As shown, the following steps are included:

[0063] In step S31, the information model of the defect information is confirmed, and self-supervised training is performed according to the information type.

[0064] In the disclosed embodiment, the information type of the obtained defect information is determined, the information model of the defect information is confirmed according to the information type, and the information model is self-supervised trained according to the information type.

[0065] The defect information model includes at least one of a visual model and a language model, and the self-supervised training can be understood as a self-supervised learning method including partial word masking prediction and next word prediction.

[0066] In step S32, based on the unimodal data obtained through self-supervised training, a visual model and a language model matching the unimodal data are obtained.

[0067] In the disclosed embodiment, unimodal data is obtained based on self-supervised training, and a model matching the unimodal data is obtained, such as a visual model and a language model.

[0068] Among them, unimodal data can be understood as data that exists in one form, such as text data or image data.

[0069] In step S33, the visual features output by the visual model and the text features output by the language model are aligned to obtain a solution.

[0070] In the disclosed embodiment, the visual model outputs visual features and the language model outputs text features, the visual features output by the visual model are converted into text features output by the language model, and then a solution is obtained based on the text features output by the language model.

[0071] In the disclosed embodiment, after self-supervised training on massive text data, the semantic encoding module can perform semantic encoding on the input text, and the encoded vector corresponding to the sentence identifier has the semantic encoding capability at the complete sentence level. According to the trained data information, the corresponding model is determined, and the visual model and language model can perform strong semantic encoding on the visual and text data, thereby obtaining a solution.

[0072] Figure 4 FIG. 1 is a flow chart showing a method of obtaining a solution according to an exemplary embodiment. Figure 4 As shown, the following steps are included:

[0073] In step S41, the visual features output by the visual model and the text features output by the language model are aligned to obtain a first alignment result.

[0074] In the disclosed embodiment, the visual features output by the visual model are converted into text features output by the language model, thereby obtaining a first alignment result after conversion, such as text information output by the language model.

[0075] Among them, the first alignment result can be understood as text information after the visual features output by the visual model are converted into text features output by the language model.

[0076] In step S42, the first alignment result is aligned with the expected data to obtain a solution.

[0077] In the disclosed embodiment, the text information expected by the user is generated according to the obtained first alignment result, and then a solution is generated according to the text information expected by the user.

[0078] The expected data may be understood as the output text expected by the user.

[0079] In the disclosed embodiment, by aligning the language model output and the user's expected output, an accurate solution to the defect problem is obtained, thereby improving the defect resolution efficiency and the defect resolution accuracy.

[0080] Figure 5 FIG. 1 is a flowchart of obtaining expected data according to an exemplary embodiment. Figure 5 As shown, the following steps are included:

[0081] In step S51, the second model is called.

[0082] In the disclosed embodiment, the second model is trained by the vertical domain data determined by the user's privatization needs, and the second model is trained according to the vertical domain data determined by the user's privatization needs.

[0083] The second model can be understood as a model including a private knowledge database.

[0084] In step S52, expected data is acquired based on the second model.

[0085] In the disclosed embodiment, according to the second model, dedicated data in the privatized knowledge database is called to obtain the desired data.

[0086] In the disclosed embodiment, by utilizing private knowledge data in vertical fields, a content index library and semantic coding library for massive data are constructed, which can quickly retrieve semantically related content according to user needs, and the model can generate decision recommendations based on the related content.

[0087] In the disclosed embodiment, a database is constructed, users input multimodal data, and the multimodal data is vector-encoded and input into a large-scale multimodal AI model together with semantically related documents. The large-scale multimodal AI model predicts the output text sequence and feeds it back to the user as a result.

[0088] Figure 6This is a schematic diagram of a full-process assisted defect management system based on a multimodal AI large model. Figure 6 The specific contents are as follows:

[0089] In the disclosed embodiment, a database is constructed.

[0090] The database can be understood as including text data in a massive database, as well as text data and image data input by users.

[0091] In the disclosed embodiment, content is extracted from massive data obtained from a massive database, and the data after content extraction is segmented into text. The segmented text data is stored in a content index library and semantically encoded, and the text data after semantic encoding is stored in a semantic encoding library.

[0092] Among them, massive data can be understood as including open domain Internet data for solving software defect management and private knowledge data in vertical fields and multimodal data, such as text data or image data.

[0093] Content extraction can be understood as extracting multimodal data information from massive data, such as text content information of documents and pictures. For picture data, the optical character recognition (OCR) interface is directly called to extract text content.

[0094] Text segmentation can be understood as dividing the text content into blocks to ensure that the length of each block remains within a moderate range without destroying the contextual semantic relationship. For example, it is generally recommended to set it within the range of 100 to 500.

[0095] The content index library can be understood as directly performing an inverted index of keywords and content block identifiers on the segmented content blocks, so that a given keyword can quickly retrieve the corresponding content block identifier and the corresponding content block text.

[0096] Semantic coding can be understood as directly calling the trained semantic coding module to vectorize the segmented text blocks, so that the encoded vectors have strong semantic relevance.

[0097] The semantic coding library can be understood as mapping and storing the content block identifier and the semantic coding library, so that the corresponding semantic coding vector can be quickly extracted from a given content identifier.

[0098] In the disclosed embodiment, the multimodal data input by the user is vector-encoded, and the semantic vector codes related to the semantic text are retrieved from the content index library and the semantic coding library respectively, and the retrieved contents are semantically sorted.

[0099] Among them, multimodal data can be understood as data in the form of text or pictures.

[0100] Vector encoding can be understood as segmenting the text data input by the user and using a semantic encoding module for vectorized encoding.

[0101] Content retrieval can be understood as quickly retrieving relevant content block identifiers and corresponding content texts from a content retrieval library based on user input after word segmentation.

[0102] Semantic sorting can be understood as extracting the corresponding semantic coding vector from the semantic coding library based on the relevant content block identifier returned by the content retrieval module, performing similarity calculation, sorting by similarity results, and selecting the top N content blocks with high similarity as the final relevant text set.

[0103] In the disclosed embodiment, the text with high correlation after semantic sorting and the data input by the user are input into a multimodal AI big model with large-scale parameters, and the AI ​​big model prediction is performed by the multimodal AI big model with large-scale parameters, and the prediction result is returned to the user as the final result.

[0104] Among them, AI big model prediction can be understood as inputting the image and text input by the user and the related text returned by the retrieval into the AI ​​big model together, and predicting the output result expected by the user. Result return can be understood as returning the predicted result to the user.

[0105] In the disclosed embodiments, the full-process assisted defect management implemented by the multimodal AI large model based on large-scale parameters will improve the production efficiency of project R&D personnel in multiple links, from defect creation, defect screening, problem location, to problem solving. It will also effectively enhance the capabilities of enterprises based on the privatized knowledge data in vertical fields, and effectively achieve experience breakthroughs by addressing the experience blind spots that exist when manually locating the root causes of project defects.

[0106] In the disclosed embodiment, the semantic encoding module is mainly responsible for vector encoding of text content, and the vector encoding has a strong semantic correlation, that is, the encoding vectors of semantically related text content have a high similarity, while the similarity between the vector encodings of semantically unrelated text content is low.

[0107] Among them, the basic unit of the neural network of the semantic coding module can adopt a temporal convolutional neural network, a recurrent neural network or a Transformer network. In the embodiment of this scheme, a Transformer network is used to build the semantic coding module.

[0108] In the disclosed embodiment, the semantic coding module training includes two stages, namely a pre-training stage and a fine-tuning stage.

[0109] Figure 7is a schematic diagram of the pre-training stage of a semantic encoding module. Figure 7 The specific contents are as follows:

[0110] In the disclosed embodiment, the input text is first segmented and a custom sentence identifier, such as [cls], is filled in the header. After semantic encoding, the model is trained using self-supervised learning loss. After self-supervised training on massive text data, the semantic encoding module can semantically encode the input text, and the encoded vector corresponding to the sentence identifier has the semantic encoding capability of the complete sentence level.

[0111] Among them, the input text can be understood as the text information after the text in the massive database is segmented, and the self-supervised learning loss can be understood as including partial word masking prediction and next word prediction.

[0112] In the disclosed embodiment, the pre-training stage can be understood as a stage of massive data and large-scale data learning, which mainly learns the initial understanding of the language.

[0113] Figure 8 is a schematic diagram of the fine-tuning phase of a semantic encoding module. Figure 8 The specific contents are as follows:

[0114] In the disclosed embodiment, after two given texts are respectively pre-trained with semantic coding modules, the model parameters are updated using the contrastive learning loss of the sentence-level semantic coding of the two texts. In the contrastive learning process, text pairs marked as semantically similar are positive samples, while other text pairs are negative samples.

[0115] Among them, input text 1 and input text 2 are defect problem information with the same meaning.

[0116] In the disclosed embodiment, the distinction of semantic coding can be improved for specific application scenarios through the fine-tuning stage.

[0117] Fig. 9 This is a schematic diagram of the pre-training phase of a large multi-modal AI model with large-scale parameters. Fig. 9 The specific contents are as follows:

[0118] In the disclosed embodiment, self-supervised training is performed on image data and text data respectively. After the visual model is divided into blocks and processed into sequence features, some visual blocks are masked and input into the visual model, and masked prediction is performed at the output end. After the text data is segmented, it is input into the language model, and the next word prediction is performed at the output end.

[0119] In the disclosed embodiment, after self-supervised training of massive image and text data, the visual model and the language model are able to perform strong semantic encoding on the visual and text data.

[0120] Fig.10 This is a schematic diagram of the fine-tuning phase of a large multi-modal AI model with large-scale parameters. Fig.10 The specific contents are as follows:

[0121] In the disclosed embodiment, the fine-tuning stage mainly realizes two capabilities of the multimodal AI large model with large-scale parameters. One capability is to align the data of the visual modality and the language modality, and the other capability is to align the language model output with the human expected output. For the first capability, in order to align the visual and language modalities, in this stage, the visual features output by the visual model and the input features of the text are spliced ​​along the time series, and then input together to the language model for training. For the second capability, in order to align the model with human expectations, it is necessary to build a multimodal instruction data set for specific tasks, that is, given the images and texts input by the user, and the related text as the instruction input, the output text expected by the user is used as the instruction data to fine-tune the model.

[0122] In the disclosed embodiment, the multimodal AI large model with large-scale parameters is mainly responsible for encoding the data after user input and content retrieval, and generating the conversation text expected by the user.

[0123] Fig.11 It is a schematic diagram of a large multi-modal AI model that uses large-scale parameters. Fig.11 The specific contents are as follows:

[0124] In the disclosed embodiment, information extraction is performed based on user feedback questions and community forums, AI assists in collecting user questions to generate structured defect data, and defects are created based on the generated structured defect data.

[0125] Among them, user feedback can be understood as the multimodal data problem that users feedback to terminal applications regarding problems they find during the process of applying the software. Community forums can be understood as open source data and corporate private knowledge databases that R&D personnel use to locate the root causes of problems based on user feedback and personal experience.

[0126] Creating defects can be understood as creating defect issues based on extracting user feedback and defect issues in community forums.

[0127] In the disclosed embodiment, defect problems are created based on open source data and data in the enterprise's private knowledge database, which expands the source of data and ensures that subsequent solutions are more targeted and accurate.

[0128] In the disclosed embodiment, based on the created defects and semantic retrieval, AI assists in labeling valid defect data and ranking by importance, thereby ensuring that defects can be screened.

[0129] Among them, semantic retrieval can be understood as data information after text segmentation and semantic encoding based on the information contained in the database, and defect screening can be understood as screening data information with high relevance to defect information based on the marked defect data and importance level sorting.

[0130] In the embodiment of the present disclosure, semantic retrieval can be used to obtain data information with high relevance to the defect problem from the database, and the data information with high relevance to the defect problem is ranked to ensure further distinction of data information with semantically similar semantics to the defect problem.

[0131] In the disclosed embodiment, based on the screened defects and intelligent questions and answers, AI assists in quickly analyzing and locating problems based on user feedback content, thereby ensuring that the problems can be located.

[0132] Among them, intelligent question and answer can be understood as analyzing and locating problems based on the screened defects using semantic relevance, and locating problems can be understood as locating problems using the results of analysis.

[0133] In the disclosed embodiments, by locating the problem, it is ensured that a solution can be quickly obtained later.

[0134] In the disclosed embodiment, based on the located problem and knowledge acquisition, AI assists in quickly acquiring technical knowledge for a specific field to troubleshoot the problem, and then uses code generation to assist in in-depth project code implementation and error review to ultimately solve the problem.

[0135] Among them, knowledge acquisition can be understood as obtaining the solution to the technical problem from the private knowledge database, and code generation can be understood as encoding the generated correct solution to obtain the solution to the problem.

[0136] In the disclosed embodiment, a large-scale multimodal AI model obtains defect problems from user feedback and community forums, and then intelligently assists project developers in the entire process to improve the efficiency of creating defects, screening defects, locating problems and solving problems based on information extraction, semantic retrieval, intelligent question and answer, knowledge acquisition and code generation capabilities.

[0137] Based on the same concept, an embodiment of the present disclosure also provides a defect information processing device.

[0138] It is understandable that in order to realize the above functions, the defect information processing device provided by the embodiment of the present disclosure includes hardware structures and / or software modules corresponding to the execution of each function. In combination with the units and algorithm steps of each example disclosed in the embodiment of the present disclosure, the embodiment of the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of the embodiment of the present disclosure.

[0139] Fig.12 FIG. 1 is a block diagram of a defect information processing device according to an exemplary embodiment. Fig.12 The device 100 includes a calling unit 101, a processing unit 102 and a generating unit 103.

[0140] The calling unit 101 is used to execute the calling of the first model, and the first model is used to perform defect management on each sub-process in the process management, and the process management includes one or more sub-processes.

[0141] The processing unit 102 is used to obtain defect information in process management based on the first model, and locate defect problems based on the defect information.

[0142] The generating unit 103 is used to generate a solution to a defect problem corresponding to any sub-process in response to locating the defect problem based on the defect information.

[0143] In one implementation, the processing unit 102 locates the defect problem based on the defect information in the following manner: semantically encodes the defect information to obtain semantic text, and based on a content index library, performs content retrieval on the semantic text to obtain a content block identifier and a content block text, wherein the content index library is used to store an index relationship between the content block identifier and the content block text; based on the content block identifier, determines the semantically encoded text corresponding to the content block identifier in the semantic encoding library; semantically sorts the semantic text according to the similarity between the content block text and the semantically encoded text; and locates the defect problem based on the semantic text after the semantic sorting.

[0144] In one embodiment, the generation unit 103 generates a solution to the corresponding defect problem in the following manner: confirm the information model of the defect information, and perform self-supervised training according to the information type, the defect information model includes at least one of a visual model and a language model; based on the unimodal data obtained by the self-supervised training, obtain a visual model and a language model that match the unimodal data; align the visual features output by the visual model and the text features output by the language model to obtain a solution.

[0145] In one embodiment, the generation unit 103 aligns the visual features output by the visual model with the text features output by the language model to obtain a solution by: aligning the visual features output by the visual model with the text features output by the language model to obtain a first alignment result; aligning the first alignment result with the expected data to obtain a solution.

[0146] In one implementation, the generation unit 103 obtains the expected data in the following manner: calling a second model, where the second model is trained with vertical domain data determined by user privatization requirements; and obtaining the expected data based on the second model.

[0147] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0148] Fig.13 2 is a block diagram of a device 200 for defect information processing according to an exemplary embodiment. For example, the device 200 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0149] Reference Fig.13 , the device 200 may include one or more of the following components: a processing component 202 , a memory 204 , a power component 206 , a multimedia component 208 , an audio component 210 , an input / output (I / O) interface 212 , a sensor component 214 , and a communication component 216 .

[0150] The processing component 202 generally controls the overall operation of the device 200, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 202 may include one or more processors 220 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 202 may include one or more modules to facilitate the interaction between the processing component 202 and other components. For example, the processing component 202 may include a multimedia module to facilitate the interaction between the multimedia component 208 and the processing component 202.

[0151] The memory 204 is configured to store various types of data to support operations on the device 200. Examples of such data include instructions for any application or method operating on the device 200, contact data, phone book data, messages, pictures, videos, etc. The memory 204 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0152] The power component 206 provides power to the various components of the device 200. The power component 206 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 200.

[0153] The multimedia component 208 includes a screen that provides an output interface between the device 200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 208 includes a front camera and / or a rear camera. When the device 200 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0154] The audio component 210 is configured to output and / or input audio signals. For example, the audio component 210 includes a microphone (MIC), and when the device 200 is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 204 or sent via the communication component 216. In some embodiments, the audio component 210 also includes a speaker for outputting audio signals.

[0155] I / O interface 212 provides an interface between processing component 202 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0156] The sensor assembly 214 includes one or more sensors for providing various aspects of the status assessment of the device 200. For example, the sensor assembly 214 can detect the open / closed state of the device 200, the relative positioning of components, such as the display and keypad of the device 200, the sensor assembly 214 can also detect the position change of the device 200 or a component of the device 200, the presence or absence of user contact with the device 200, the orientation or acceleration / deceleration of the device 200 and the temperature change of the device 200. The sensor assembly 214 can include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 214 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 214 can also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

[0157] The communication component 216 is configured to facilitate wired or wireless communication between the device 200 and other devices. The device 200 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 216 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 216 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0158] In an exemplary embodiment, the apparatus 200 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0159] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 204 including instructions, and the instructions can be executed by the processor 220 of the device 200 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0160] It is to be understood that in the present disclosure, "plurality" refers to two or more than two, and other quantifiers are similar. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. The singular forms "a", "the" and "the" are also intended to include plural forms, unless the context clearly indicates other meanings.

[0161] It is further understood that the terms "first", "second", etc. are used to describe various information, but such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other, and do not indicate a specific order or degree of importance. In fact, the expressions "first", "second", etc. can be used interchangeably. For example, without departing from the scope of the present disclosure, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information.

[0162] It will be further understood that the terms “center”, “longitudinal”, “lateral”, “front”, “back”, “up”, “down”, “left”, “right”, “vertical”, “horizontal”, “top”, “bottom”, “inside”, “outside”, etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the present embodiment and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation.

[0163] It can be further understood that, unless otherwise specified, “connection” includes a direct connection without other components between the two, and also includes an indirect connection with other components between the two.

[0164] It is further understood that, although the operations are described in a specific order in the drawings in the embodiments of the present disclosure, it should not be understood as requiring the operations to be performed in the specific order shown or in a serial order, or requiring the execution of all the operations shown to obtain the desired results. In certain environments, multitasking and parallel processing may be advantageous.

[0165] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modifications, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure.

[0166] It should be understood that the present disclosure is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the scope of the appended claims.

Claims

1. A defect information processing method, characterized in that: The method comprises: Calling a first model, wherein the first model is used to perform defect management on each sub-process in a process management, wherein the process management includes one or more sub-processes; Acquire defect information in process management based on the first model, and locate defect problems based on the defect information; In response to locating a defect problem based on the defect information for any sub-process, a solution corresponding to the defect problem is generated.

2. The method according to claim 1, characterized in that The locating the defect problem based on the defect information includes: Semantically encode the defect information to obtain semantic text, and perform content retrieval on the semantic text based on a content index library to obtain a content block identifier and a content block text, wherein the content index library is used to store an index relationship between the content block identifier and the content block text; Based on the content block identifier, determining a semantically coded text corresponding to the content block identifier in a semantic coding library; Semantically sorting the semantic text according to the similarity between the content block text and the semantically coded text; Locate defect problems based on semantic text after semantic sorting.

3. The method according to claim 1, characterized in that The defect information includes multimodal data, and the generating a solution corresponding to the defect problem includes: Confirming an information model of the defect information, and performing self-supervised training according to the information type, the defect information model comprising at least one of a visual model and a language model; Based on the unimodal data obtained by the self-supervised training, a visual model and a language model matching the unimodal data are obtained; The visual features output by the visual model and the text features output by the language model are aligned to obtain a solution.

4. The method according to claim 3, characterized in that The aligning of the visual features output by the visual model and the text features output by the language model to obtain a solution includes: Aligning the visual features output by the visual model with the text features output by the language model to obtain a first alignment result; The first alignment result is aligned with expected data to obtain a solution.

5. The method according to claim 4, characterized in that The obtaining of expected data comprises: Calling a second model, where the second model is trained by vertical domain data determined by the user's privatization requirements; Based on the second model, expected data is obtained.

6. A defect information processing device, characterized in that: include: A calling unit, used to execute and call a first model, wherein the first model is used to perform defect management on each sub-process in a process management, wherein the process management includes one or more sub-processes; A processing unit, configured to acquire defect information in process management based on the first model, and locate defect problems based on the defect information; The generating unit is used for locating a defect problem based on the defect information in response to any sub-process, and generating a solution corresponding to the defect problem.

7. The defect information processing device according to claim 6, characterized in that: The processing unit locates the defect problem based on the defect information in the following manner: Semantically encode the defect information to obtain semantic text, and perform content retrieval on the semantic text based on a content index library to obtain a content block identifier and a content block text, wherein the content index library is used to store an index relationship between the content block identifier and the content block text; Based on the content block identifier, determining a semantically coded text corresponding to the content block identifier in a semantic coding library; Semantically sorting the semantic text according to the similarity between the content block text and the semantically coded text; Locate defect problems based on semantic text after semantic sorting.

8. The defect information processing device according to claim 6, characterized in that: The generating unit generates a solution corresponding to the defect problem in the following manner: Confirming an information model of the defect information, and performing self-supervised training according to the information type, the defect information model comprising at least one of a visual model and a language model; Based on the unimodal data obtained by the self-supervised training, a visual model and a language model matching the unimodal data are obtained; The visual features output by the visual model and the text features output by the language model are aligned to obtain a solution.

9. The defect information processing device according to claim 8, characterized in that: The generating unit aligns the visual features output by the visual model and the text features output by the language model in the following manner to obtain a solution: Aligning the visual features output by the visual model with the text features output by the language model to obtain a first alignment result; The first alignment result is aligned with expected data to obtain a solution.

10. The defect information processing device according to claim 9, characterized in that: The generating unit obtains the expected data in the following manner: Calling a second model, where the second model is trained by vertical domain data determined by the user's privatization requirements; Based on the second model, expected data is obtained.

11. A defect information processing device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the method described in any one of claims 1 to 5.

12. A storage medium, characterized in that: The storage medium stores instructions, and when the instructions in the storage medium are executed by a processor of the terminal, the terminal is enabled to execute the method described in any one of claims 1 to 5.

Images