Data retrieval method and device for defect analysis, computer device and medium

Abstract

The present application relates to the technical field of information retrieval, and in particular to a data retrieval method and device for defect analysis, a computer device and a medium. The method screens basic fields in historical defect data to obtain a basic field set, extracts subject fields of defect description information to obtain a subject field set, deletes fields intersecting the basic field set and the subject field set from the subject field set to obtain a target field set, clusters all subject fields in the target field set into at least one cluster set, calculates reference similarity of each cluster set with the basic field set respectively, calculates the average of similarity of a field to be retrieved with all basic fields, and determines the cluster set corresponding to the reference similarity closest to the average as a retrieval result. When the basic field library is updated, the retrieval content can be dynamically adjusted, the accuracy of data retrieval is improved, and the maintainability and reliability of the financial service platform are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of information retrieval, and in particular to a data retrieval method and device for defect analysis, a computer device and a medium. BACKGROUND

[0002] With the development of science and technology, a plurality of software-based service functions can be expanded in a financial service platform. The financial service platform can be an insurance system, a bank system, a transaction system, an order system, etc. The financial service platform can support functions such as shopping, socializing, interactive gaming, and resource transfer. It can also have functions such as applying for a loan, a credit card, or purchasing insurance, financial products, etc.

[0003] However, in the software development process involving financial software, software defects are often unavoidable, which can reduce the service reliability of the financial service platform. Current ways to reduce defects can be divided into direct and indirect ways. Direct ways include software testing, standardizing development processes, enhancing development architecture design, and improving developer skills. Indirect ways usually refer to analyzing and summarizing historical defect data to improve the ability to avoid defects in the development process.

[0004] However, current analysis and summary of historical defect data usually involves manual annotation and classification by testers or annotation and classification based on text classification algorithms. Then, the database is searched based on the annotated categories. However, as defect data continues to increase, the accuracy of historical annotation categories may decrease. Moreover, a large amount of historical defect data may be included in the same annotation category dimension, making it difficult to accurately obtain historical defect data associated with the data to be searched, resulting in low accuracy of data retrieval. Therefore, how to improve the accuracy of data retrieval has become a problem to be solved. SUMMARY

[0005] Therefore, the embodiments of the present application provide a data retrieval method and device for defect analysis, a computer device and a medium to solve the problem of data retrieval accuracy.

[0006] In a first aspect, the embodiments of the present application provide a data retrieval method for defect analysis, which comprises:

[0007] Obtain historical defect data, and use a preset basic field library to perform basic field filtering on the historical defect data to obtain a basic field set containing at least one basic field;

[0008] Obtain defect description information of the historical defect data, and perform subject field extraction on the defect description information to obtain a subject field set composed of at least one subject field;

[0009] The base field set and the theme field set are compared, intersecting fields are determined, the intersecting fields are deleted from the theme field set, and a target field set is obtained;

[0010] All theme fields in the target field set are clustered to obtain at least one cluster set, and a reference similarity of each cluster set with the base field set is calculated to obtain a corresponding reference similarity of the cluster set;

[0011] A field to be searched is obtained, an average of similarities of the field to be searched with all base fields is calculated, and a cluster set corresponding to a reference similarity closest to the average is determined as a search result of the field to be searched.

[0012] In a second aspect, an embodiment of the present application provides a data search device for defect analysis, and the data search device comprises:

[0013] A field screening module is configured to obtain historical defect data, perform base field screening on the historical defect data by using a preset base field library, and obtain a base field set comprising at least one base field.

[0014] A field extraction module is configured to obtain defect description information of the historical defect data, perform theme field extraction on the defect description information, and obtain a theme field set comprising at least one theme field.

[0015] A field comparison module is configured to compare the base field set and the theme field set, determine intersecting fields, delete the intersecting fields from the theme field set, and obtain a target field set.

[0016] A field clustering module is configured to cluster all theme fields in the target field set to obtain at least one cluster set, calculate a similarity of each cluster set with the base field set, and obtain a reference similarity of a corresponding cluster set.

[0017] A field search module is configured to obtain a field to be searched, calculate an average of similarities of the field to be searched with all base fields, and determine a cluster set corresponding to a reference similarity closest to the average as a search result of the field to be searched.

[0018] In a third aspect, an embodiment of the present application provides a computer device, which comprises a processor, a memory, and a computer program stored in the memory and capable of running on the processor, and the processor implements the data search method according to the first aspect when executing the computer program.

[0019] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the data retrieval method according to the first aspect.

[0020] Compared with the prior art, the embodiment of the present application has the following beneficial effects:

[0021] The historical defect data is acquired, a preset basic field library is used to perform basic field screening on the historical defect data, a basic field set containing at least one basic field is obtained, defect description information of the historical defect data is acquired, subject field extraction is performed on the defect description information, a subject field set composed of at least one subject field is obtained, the basic field set and the subject field set are compared, the intersecting fields are determined, the intersecting fields are deleted from the subject field set, a target field set is obtained, clustering processing is performed on all subject fields in the target field set, at least one cluster set is obtained, the similarity of each cluster set to the basic field set is calculated, the reference similarity corresponding to each cluster set is obtained, a field to be retrieved is acquired, the average of the similarity of the field to be retrieved to all basic fields is calculated, and the cluster set corresponding to the reference similarity closest to the average is determined as the retrieval result of the field to be retrieved. When the basic field library is updated, the dynamic adjustment of the retrieval strategy can be realized, the potential association between the subject fields is mined in a clustering manner, the subject field set with strong association is provided as the retrieval result for the data to be retrieved in combination with the similarity calculation result, the accuracy of data retrieval is improved, the accuracy and efficiency of defect analysis of the data to be retrieved based on the retrieval result are improved, and the maintainability and reliability of software in the financial service platform are improved. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0023] Figure 1 is an application environment schematic diagram of a data retrieval method for defect analysis provided by the first embodiment of the present application;

[0024] Figure 2 is a flowchart of a data retrieval method for defect analysis provided by the first embodiment of the present application;

[0025] Figure 3 is a structure schematic diagram of a data retrieval device for defect analysis provided by the second embodiment of the present application;

[0026] Figure 4 is a structural schematic diagram of a computer device provided by Embodiment Three of the present application. DETAILED DESCRIPTION

[0027] In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular sequences of steps, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known methods, devices, circuits, and

[0028] It is to be understood that the terminology "includes", "has", "holds", "contains" and / or "comprising", "including", "containing", "having" and / or "comprises" when used in the present specification and in the accompanying claims, specifies the presence of stated features, integers, steps, operations, elements, and / or components but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0029] It is also to be understood that the terminology "and / or" where used in the present specification and in the accompanying claims, is used to describe one or more of the stated features, integers, steps, operations, elements, components and / or combinations thereof.

[0030] As used in the present specification and in the accompanying claims, the term "if' can be interpreted as meaning "when" or "once" or "in response to a determination" or "in response to a detection" depending on the context. Similarly, the phrase "if determined" or "if detected [the described condition or event]" can be interpreted as meaning "once determined" or "in response to a determination" or "once detected [the described condition or event]" or "in response to a detection [the described condition or event]" depending on the context.

[0031] In addition, the terms "first", "second", "third", etc. in the description of the present application and in the accompanying claims are only used to distinguish descriptions, and cannot be understood as indicating or implying relative importance.

[0032] Reference in the specification to "one embodiment" or "some embodiments" etc., means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. The appearances of the phrases "in one embodiment", "in some embodiments", "in other embodiments", "in additional embodiments", etc., in various places in the specification are not necessarily all referring to the same embodiment, unless otherwise specifically stated. The terms "comprise", "comprising", "have", "having", "include", "including", "contain", "containing", and the like, are used in the sense of "including but not limited to", unless otherwise specifically stated.

[0033] The embodiments of the present application can acquire and process related data based on artificial intelligence technology. The artificial intelligence (AI) is a theory, method, technology and application system for using a digital computer or a machine controlled by a digital computer to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain optimal results.

[0034] The artificial intelligence basic technology generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. The artificial intelligence software technology mainly includes computer vision technology, robot technology, biological retrieval technology, speech processing technology, natural language processing technology and machine learning / deep learning, etc.

[0035] It should be understood that the size of the serial number of each step in the following embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0036] In order to illustrate the technical solutions of the present application, the following will be explained by specific embodiments.

[0037] The data retrieval method for defect analysis provided by the embodiment of the present application can be applied in the application environment such as Figure 1 , wherein the client and the server communicate. The client includes but is not limited to a palm computer, a desktop computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cloud terminal device, a personal digital assistant (PDA) and other computer devices. The server can be a stand-alone server, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution networks (CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

[0038] Client and server sides can be deployed within a financial service platform. Financial service platforms typically extend to multiple software-based service functions. These platforms can be insurance systems, banking systems, transaction systems, order systems, etc. They can support functions such as shopping, social networking, interactive games, and resource transfer. They can also provide functions such as loan applications, credit card applications, or the purchase of insurance and wealth management products. The client side can provide data retrieval services and perform defect analysis on the software involved in the extended functions within the financial service platform, thereby improving the maintainability and reliability of the platform.

[0039] See Figure 2 This is a flowchart illustrating a data retrieval method for defect analysis provided in Embodiment 1 of the present invention. The data retrieval method described above can be applied to... Figure 1 The client-side computer connects to the server to retrieve stored historical defect data and its descriptions. The server may store a pre-defined base field library, which provides reference data for the base field filtering process. The client-side computer interacts with the user to obtain the user-provided fields to be searched; these fields are the ones from which data retrieval is performed. Figure 2 As shown, the data retrieval method may include the following steps:

[0040] Step S201: Obtain historical defect data, and use a preset basic field library to filter the historical defect data by basic fields to obtain a set of basic fields containing at least one basic field.

[0041] Historical defect data can be defect data that has been identified in a historical period. In this embodiment, historical defect data can refer to defect data related to financial service software within the financial service platform. The basic field library can include at least one basic field, and the basic field set can include at least one basic field selected from the basic field library. The basic field can include defect type, defect cause, etc.

[0042] Specifically, a historical time period can refer to the time interval between a preset historical time point and the current time point. In one implementation, the implementer can also preset two different historical time points to determine the historical time period.

[0043] Optionally, the basic field library contains N basic fields, and the historical defect data contains at least one data field;

[0044] Historical defect data is filtered using a pre-defined basic field library to obtain a set of basic fields containing at least one basic field, including:

[0045] Initialize the basic field set as an empty set, and match the basic field with each data field contained in the historical defect data for any basic field in the basic field library. If the basic field matches any data field successfully, the basic field is added to the basic field set.

[0046] All basic fields in the basic field library are traversed to obtain a basic field set containing at least one basic field.

[0047] Wherein, N is an integer greater than zero, and the data field can refer to all fields obtained by field segmentation of the historical defect data.

[0048] Specifically, the field segmentation can adopt a separator-based segmentation manner, and the separator can include a comma, a period, a semicolon, a space, etc.

[0049] After initialization, the basic field set is an empty set, i.e., does not contain any basic field. Each basic field in the basic field library is matched with each data field contained in the historical defect data. When the basic field is the same as the data field, it means that the matching is successful, and all matching successful basic fields are added to the basic field set.

[0050] In an embodiment, since the same meaning different words may occur during field matching, the similarity of the basic field and the data field is compared with the preset matching threshold to determine whether the matching is successful. When the similarity of the basic field and the data field is greater than the preset matching threshold, it means that the matching is successful. When the similarity of the basic field and the data field is less than or equal to the preset matching threshold, it means that the matching fails. Further, in order to improve the attention degree of the semantic information in the similarity calculation process, a trained semantic feature extraction model can be used to extract the basic field semantic feature and the data field semantic feature of the basic field and the data field, respectively, and then the similarity is calculated according to the basic field semantic feature and the data field semantic feature. The trained semantic feature extraction model can use the encoder structure of the trained Transformer model, the encoder structure of the trained word vector conversion model, etc.

[0051] In this embodiment, the extraction of the basic field in the historical defect data is performed according to the basic field library. The basic field library can be dynamically updated to adapt to newly appeared defect data, and the dynamic adjustment of the subsequent retrieval strategy can be realized, thereby improving the accuracy of data retrieval.

[0052] The above step of obtaining the historical defect data, filtering the basic field of the historical defect data by using the preset basic field library, and obtaining a basic field set containing at least one basic field extracts the basic field from the historical defect data, provides reference information for subsequent data retrieval, and thereby improves the accuracy of data retrieval.

[0053] In step S202, defect description information of historical defect data is acquired, and subject field extraction is performed on the defect description information to obtain a subject field set composed of at least one subject field.

[0054] The defect description information can be detailed description of causes and defect phenomenon description of the historical defect data, the subject field can be a field capable of representing main information of the defect description information, and the subject field set includes at least one subject field.

[0055] Specifically, the defect description information contains more field information than the historical defect data, and therefore needs to be simplified, that is, subject field extraction is performed. Before the subject field extraction, word segmentation processing is performed on the defect description information to obtain a plurality of description fields. The word segmentation processing can be performed in a language model-based manner or a statistical learning-based manner, for example, a hidden Markov model or a conditional random field model.

[0056] Optionally, the defect description information includes at least one description field.

[0057] The subject field extraction is performed on the defect description information to obtain a subject field set composed of at least one subject field, including:

[0058] For any description field, the number of occurrences of the description field in the defect description information is counted to obtain a target number, the total number of occurrences of all description fields in the defect description information is acquired to obtain a total number, and a first ratio of the target number and the total number is calculated, and the first ratio is taken as the term frequency of the description field.

[0059] All reference description information included in a preset database is acquired, the number of reference description information including the description field is counted to obtain a target number, the number of all reference description information is acquired to obtain a total number, and a second ratio of the total number and the target number is calculated, and the logarithm of the second ratio is taken as the inverse frequency of the description field.

[0060] The term and the inverse frequency are multiplied to determine the importance of the description field, and the importance of the corresponding description field is obtained by traversing all description fields.

[0061] The description field with an importance greater than a preset threshold is determined as a subject field, and the subject field set is formed by all subject fields.

[0062] The description field can be field information included in defect description information, all description fields can constitute defect description information, the target number of times can represent the number of times of the description field in the defect description information, the total number of times can represent the number of all description fields in the defect description information, that is, the sum of the number of times of all description fields in the defect description information, the first ratio can be the result of the target number of times divided by the total number of times, and the word frequency can represent the frequency of the description field in the defect description information.

[0063] The reference description information can be defect description information corresponding to other historical defect data, the database can include at least one reference description information, the target number can be the number of reference description information including the description field, the total number can be the number of all reference description information, the first ratio can be the result of the total number divided by the target number, and the inverse frequency can represent the general importance of the field.

[0064] The importance degree can be used to measure the information representation ability of the description field for the defect description information, and the preset threshold can be used to determine whether the importance degree of the description field is sufficient to be the theme field.

[0065] Specifically, the higher the word frequency, the more times the description field appears in the defect description information, and the greater the inverse frequency, the stronger the class distinguishing ability of the description field, that is, the ability to distinguish the defect description information from the reference description information. The importance degree of the description field can be determined by combining the word frequency and the inverse frequency, that is, the representation ability of the description field for the core semantics of the defect description information.

[0066] In the embodiment, the importance degree of the description field is calculated by the word frequency and the inverse frequency, which is simple and fast, and can effectively extract the theme field, thereby improving the efficiency and accuracy of subsequent data retrieval.

[0067] Optionally, the description field with an importance degree greater than a preset threshold is determined as a theme field, and a theme field set is formed by all theme fields, including:

[0068] The number of all description fields is counted to obtain a field number, the field number is multiplied by a preset coefficient, and the multiplication result is determined as a field identifier;

[0069] All description fields are sorted in descending order of importance degree to obtain a field sequence, and one description field corresponds to a field serial number in the field sequence;

[0070] The reference description field corresponding to the field serial number closest to the field identifier in the field sequence is determined, the importance degree corresponding to the reference description field is taken as a preset threshold, and the description field with an importance degree greater than the preset threshold is determined as a theme field. A theme field set is formed by all theme fields.

[0071] wherein the number of fields can refer to the total number of all description fields, the preset coefficient can have a value range of [0, 1], the field identifier can refer to the position identifier of the field, the field sequence can include all description fields, each description field has a corresponding field serial number in the field sequence, the field serial number can represent the corresponding position of the description field in the field sequence, and the reference description field can refer to the description field used to determine the preset threshold.

[0072] Specifically, in the embodiment, the preset coefficient can be 0.5, that is, the description field at the middle position in the field sequence is taken as the reference description field, and accordingly, that is, the half of the description fields ranked in the front in the field sequence are determined as the theme fields.

[0073] In the embodiment, the theme fields are determined by the sorting and screening, which can effectively eliminate some description fields with low importance, improve the representation ability of the theme field set composed of the theme fields for the defect description information, and thus improve the accuracy of subsequent data retrieval.

[0074] The steps of obtaining the defect description information of the historical defect data, extracting the theme fields from the defect description information, and obtaining the theme field set composed of at least one theme field, can represent the defect description information by the theme field set, screen out irrelevant description fields, and thus reduce the calculation amount and improve the accuracy and efficiency of the subsequent data retrieval process.

[0075] In step S203, the base field set and the theme field set are compared to determine the intersecting fields, the intersecting fields are deleted from the theme field set, and a target field set is obtained.

[0076] wherein the intersection can refer to the fields existing in both the base field set and the theme field set, and the target field set can refer to the theme field set after the intersecting fields are deleted.

[0077] Specifically, when there are intersecting fields, it indicates that the contents of the base field set and the theme field set are repeated, and thus the intersecting fields are deleted from the theme field set to obtain the target field set.

[0078] The steps of comparing the base field set and the theme field set to determine the intersecting fields, deleting the intersecting fields from the theme field set, and obtaining the target field set, can eliminate the repeated fields and improve the efficiency of subsequent data retrieval.

[0079] In step S204, all theme fields in the target field set are clustered to obtain at least one cluster set, the similarity of each cluster set with the base field set is calculated to obtain the reference similarity of the corresponding cluster set.

[0080] The reference similarity can be used to represent semantic similarity between the cluster set and the base field set.

[0081] Specifically, the clustering processing can be calculated according to the similarity between the topic fields, and the similarity between the topic fields can be calculated by distance measurement. The similarity between the cluster set and the base field set can be calculated by using the intersection-union ratio method, and the calculation efficiency of the intersection-union ratio method is higher.

[0082] In an embodiment, the similarity between the cluster set and the base field set can also be calculated by using the distance measurement method.

[0083] Optionally, the similarity between each cluster set and the base field set is calculated to obtain the reference similarity of the corresponding cluster set, including:

[0084] For any cluster set, the similarity between each topic field in the cluster set and all base fields in the base field set is calculated to obtain the first average value of the corresponding topic field.

[0085] The first average values of all topic fields are averaged to obtain a second average value, and the second average value is determined as the reference similarity of the cluster set.

[0086] All cluster sets are traversed to obtain the reference similarity of the corresponding cluster set.

[0087] The first average value can be the average value of the similarity between a topic field and all base fields, and the second average value can be the average calculation result of all first average values.

[0088] In this embodiment, the similarity between each topic field in the cluster set and all base fields in the base field set is calculated, and then the second average value is determined as the reference similarity according to the calculation result, so that the reference similarity can be obtained more accurately, which is convenient for subsequent similarity comparison and improves the accuracy of data retrieval.

[0089] Optionally, the similarity between each topic field in the cluster set and all base fields in the base field set is calculated to obtain the first average value of the corresponding topic field, including:

[0090] For any topic field in the cluster set, a topic field embedding vector of the topic field is extracted by using a pre-trained word embedding model.

[0091] The base field embedding vector corresponding to each base field in the base field set is extracted by using the word embedding model.

[0092] The vector similarity between the topic field embedding vector and each base field embedding vector is calculated, the vector similarity corresponding to each base field vector is obtained, the mean value of the vector similarities of all base field vectors is calculated, the first mean value of the topic field is obtained, and the first mean value of the corresponding topic field is obtained by traversing all topic fields in the cluster set.

[0093] The pre-trained word embedding model can be a Word2Vec model, a Transformer model, etc., the topic field embedding vector can be used to represent the semantic information of the topic field, and the base field embedding vector can be used to represent the semantic information of the base field.

[0094] The vector similarity can refer to the similarity between the topic field embedding vector and the base field embedding vector.

[0095] Specifically, the vector similarity can be calculated by using a distance measurement method such as Euclidean distance or cosine similarity.

[0096] In this embodiment, the semantic information of the field is represented by vector embedding, so that the similarity can be calculated based on the semantic information, avoiding the case that the similarity is low in the case of synonymous different words, and more meeting the needs of field comparison, thereby further improving the accuracy of subsequent data retrieval.

[0097] The above step of clustering all topic fields in the target field set to obtain at least one cluster set and calculating the similarity between each cluster set and the base field set to obtain the reference similarity corresponding to each cluster set provides a reference similarity for the subsequent retrieval process, thereby improving the efficiency of data retrieval.

[0098] In step S205, a field to be retrieved is obtained, the similarity between the field to be retrieved and all base fields is calculated, and the cluster set corresponding to the reference similarity closest to the mean value is determined as the retrieval result of the field to be retrieved.

[0099] The field to be retrieved can refer to a field that needs to be retrieved for defect information, and the retrieval result can refer to the topic field in the cluster set obtained by retrieval.

[0100] Specifically, after obtaining the retrieval result, the corresponding defect description information can be determined according to the topic field in the cluster set obtained by retrieval, thereby providing sufficient similar defect data for defect analysis of the field to be retrieved.

[0101] In an embodiment, the cluster set corresponding to the Q reference similarities closest to the mean value can be determined as the retrieval result, and Q is a positive integer, for example, Q can be 3.

[0102] Optionally, the historical defect data comprises a historical defect type, and the data to be searched comprises a defect type to be searched;

[0103] The average of the similarity between the field to be searched and all the base fields is calculated, comprising:

[0104] The historical defect type and the defect type to be searched are compared to obtain a comparison result;

[0105] If the comparison result is that the historical defect type and the defect type to be searched are the same, the average of the similarity between the field to be searched and all the base fields in the base field set is calculated.

[0106] The historical defect type can refer to the defect type to which the historical defect data belongs, the defect type to be searched can refer to the defect type for which the data to be searched needs to be searched, and the comparison result can include that the historical defect type and the defect type to be searched are the same and that the historical defect type and the defect type to be searched are different.

[0107] Specifically, the defect type to be searched can be set by an implementer to determine the historical defect data of which the defect type needs to be searched by the data to be searched. It should be noted that the same data to be searched can include multiple defect types to be searched, at which time multiple data searches need to be performed for different defect types to be searched, and one historical defect data corresponds to one historical defect type.

[0108] In this embodiment, by comparing the historical defect type and the defect type to be searched, the comparison condition of the calculation process is set, so that only the data of a certain defect type can be searched, the search object is greatly reduced, the search efficiency is improved, and the expected type of search data can be provided, and the accuracy of the search is improved.

[0109] The above steps of obtaining the field to be searched, calculating the average of the similarity between the field to be searched and all the base fields, and determining the cluster set corresponding to the reference similarity closest to the average as the search result of the field to be searched, by respectively searching the similarity average of the base field, the cluster set with similar semantics is obtained as the search result, the relevance of the search result and the field to be searched is improved, that is, the accuracy of the search is improved.

[0110] In this embodiment, when the base field library is updated, the dynamic adjustment of the search strategy can be realized, the potential association between the theme fields is mined by clustering, and the theme field set with strong relevance is provided as the search result for the data to be searched based on the similarity calculation result, the accuracy of the data search is improved, the accuracy and efficiency of the defect analysis of the data to be searched based on the search result are improved, and the maintainability and reliability of the software in the financial service platform are improved.

[0111] The data retrieval method for defect analysis corresponding to the above embodiment, Figure 3 A structural block diagram of the data retrieval device for defect analysis provided by the second embodiment of the present application is shown. The data retrieval device is applied to a client. A computer device corresponding to the client is connected to a server to obtain stored historical defect data and defect description information thereof from the server. The server can store a preset basic field library. The basic field library can provide reference data for a basic field screening process. The computer device corresponding to the client can interact with a user to obtain a to-be-retrieved field provided by the user. The to-be-retrieved field is a field that needs to be retrieved. For ease of illustration, only parts related to the embodiments of the present application are shown.

[0112] Referring to Figure 3 The data retrieval device comprises:

[0113] The field screening module 31 is configured to obtain historical defect data, perform basic field screening on the historical defect data by using a preset basic field library, and obtain a basic field set containing at least one basic field.

[0114] The field extraction module 32 is configured to obtain defect description information of the historical defect data, perform subject field extraction on the defect description information, and obtain a subject field set composed of at least one subject field.

[0115] The field comparison module 33 is configured to compare the basic field set and the subject field set, determine an intersecting field, delete the intersecting field from the subject field set, and obtain a target field set.

[0116] The field clustering module 34 is configured to perform clustering processing on all subject fields in the target field set, obtain at least one clustering set, calculate a similarity degree between each clustering set and the basic field set, and obtain a reference similarity degree corresponding to the clustering set.

[0117] The field retrieval module 35 is configured to obtain a to-be-retrieved field, calculate an average value of similarity degrees between the to-be-retrieved field and all basic fields, and determine a clustering set corresponding to a reference similarity degree closest to the average value as a retrieval result of the to-be-retrieved field.

[0118] Optionally, the basic field library contains N basic fields, the historical defect data contains at least one data field, and N is an integer greater than zero.

[0119] The field screening module 31 comprises:

[0120] The set initialization unit is configured to initialize the basic field set as an empty set, and match each data field contained in the historical defect data with any basic field in the basic field library, and if the basic field matches any data field, the basic field is added to the basic field set;

[0121] The field traversal unit is configured to traverse all the basic fields in the basic field library to obtain the basic field set containing at least one basic field.

[0122] Optionally, the defect description information contains at least one description field.

[0123] The field extraction module 32 includes:

[0124] The word frequency calculation unit is configured to count the number of times of occurrence of the description field in the defect description information to obtain a target number of times, count the total number of times of occurrence of all the description fields in the defect description information to obtain a total number of times, calculate a first ratio of the target number of times and the total number of times, and take the first ratio as the word frequency of the description field.

[0125] The inverse frequency calculation unit is configured to obtain all the reference defect description information contained in the preset database, count the number of the reference defect description information containing the description field to obtain a target number, count the total number of all the reference defect description information to obtain a total number, calculate a second ratio of the total number and the target number, and take the logarithm of the second ratio as the inverse frequency of the description field.

[0126] The importance determination unit is configured to multiply the word frequency and the inverse frequency to determine the multiplication result as the importance of the description field, and traverse all the description fields to obtain the importance of the corresponding description field.

[0127] The threshold comparison unit is configured to determine the description field with the importance greater than a preset threshold as a theme field, and form a theme field set from all the theme fields.

[0128] Optionally, the threshold comparison unit includes:

[0129] The identifier determination subunit is configured to count the number of all the description fields to obtain a field number, multiply the field number and a preset coefficient to determine the multiplication result as the field identifier.

[0130] The field sorting subunit is configured to sort all the description fields in descending order of the importance to obtain a field sequence, and one description field corresponds to one field serial number in the field sequence.

[0131] The threshold determination subunit is configured to determine a reference description field corresponding to a field sequence number closest to the field identification in the field sequence, take an importance degree corresponding to the reference description field as a preset threshold, and determine a description field with an importance degree greater than the preset threshold as a theme field, so as to form a theme field set by all theme fields.

[0132] Optionally, the field clustering module 34 includes:

[0133] The first mean value calculation unit is configured to calculate a similarity mean value of each theme field in the cluster set and all base fields in the base field set, so as to obtain a first mean value of the corresponding theme field.

[0134] The second mean value calculation unit is configured to calculate a mean value of the first mean values of all theme fields, so as to obtain a second mean value, and determine the second mean value as a reference similarity of the cluster set.

[0135] The set traversal unit is configured to traverse all cluster sets, so as to obtain the reference similarity of the corresponding cluster set.

[0136] Optionally, the first mean value calculation unit includes:

[0137] The first vector embedding subunit is configured to extract a theme field embedding vector of the theme field by using a pre-trained word embedding model for any theme field in the cluster set.

[0138] The second vector embedding subunit is configured to extract a base field embedding vector corresponding to each base field in the base field set by using the word embedding model.

[0139] The similarity calculation subunit is configured to calculate a vector similarity between the theme field embedding vector and each base field embedding vector, so as to obtain a vector similarity of the corresponding base field vector, calculate a mean value of the vector similarities of all base field vectors, so as to obtain the first mean value of the theme field, and traverse all theme fields in the cluster set, so as to obtain the first mean value of the corresponding theme field.

[0140] Optionally, the historical defect data includes a historical defect type, and the data to be searched includes a defect type to be searched.

[0141] The field searching module 35 includes:

[0142] The type comparison unit is configured to compare the historical defect type and the defect type to be searched, so as to obtain a comparison result.

[0143] The calculation execution unit is configured to calculate a mean value of the similarity between the field to be searched and all base fields in the base field set if the comparison result is that the historical defect type and the defect type to be searched are the same.

[0144] It should be noted that the information interaction and execution process between the above modules, units, and sub-units are based on the same concept as the method embodiments of the present invention. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0145] Figure 4 This is a schematic diagram of the structure of a computer device provided in Embodiment 3 of the present invention. Figure 4 As shown, the computer device of this embodiment includes: at least one processor ( Figure 4 Only one is shown in the diagram), a memory, and a computer program stored in the memory and executable on at least one processor, which, when executed by the processor, implements the steps in any of the above-described data retrieval method embodiments.

[0146] This computer device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 4 The examples of computer devices are merely examples and do not constitute a limitation on computer devices. Computer devices may include more or fewer components than shown in the illustration, or combinations of certain components, or different components, such as network interfaces, displays, and input devices.

[0147] The processor referred to can be a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0148] The memory includes a readable storage medium, an internal memory, etc., wherein the internal memory can be a memory of the computer device, and the internal memory provides an environment for running of the operating system and the computer-readable instructions in the readable storage medium. The readable storage medium can be a hard disk of the computer device, and in other embodiments, can also be an external storage device of the computer device, for example, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Further, the memory can include both an internal storage unit of the computer device and an external storage device. The memory is used to store an operating system, an application program, a BootLoader, data, and other programs, such as program codes of computer programs, etc. The memory can also be used to temporarily store data that has been output or will be output.

[0149] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit. In addition, the specific names of each functional unit and module are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of the unit and module in the above device can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here. If the integrated unit is realized in the form of software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the present application realizes all or part of the processes in the above-mentioned embodiment methods, which can be completed by a computer program to instruct related hardware. The computer program can be stored in a computer readable storage medium, and when the processor executes the computer program, the steps of the above-mentioned method embodiment can be realized. The computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form, etc. The computer readable medium can at least include any entity or device capable of carrying computer program code, recording medium, computer memory, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), electrical carrier signal, telecommunication signal and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc. In some jurisdictions, according to legislation and patent practice, the computer readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0150] The present application realizes all or part of the processes in the above-mentioned embodiment methods, which can also be completed by a computer program product. When the computer program product runs on the computer device, it makes the computer device execute the steps in the above-mentioned method embodiment.

[0151] In the above-mentioned embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0152] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0153] In the embodiments provided by the present application, it should be understood that the disclosed apparatus / computer device and method can be implemented in other manners. For example, the described apparatus / computer device embodiments are merely schematic. For example, the division of the modules or units can be different, and each can include a plurality of sub-units. Some or all of the modules or units can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0154] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e. may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0155] The above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A data retrieval method for defect analysis, characterized in that, The data retrieval method includes: Acquire historical defect data, and use a preset basic field library to filter the historical defect data to obtain a basic field set containing at least one basic field. Obtain the defect description information of the historical defect data, extract the topic fields from the defect description information, and obtain a topic field set consisting of at least one topic field; The basic field set and the topic field set are compared to determine the intersecting fields. The intersecting fields are then deleted from the topic field set to obtain the target field set. Clustering is performed on all topic fields in the target field set to obtain at least one cluster set. The similarity between each cluster set and the base field set is calculated to obtain the reference similarity of the corresponding cluster set. The calculation of the similarity between each cluster set and the base field set to obtain the reference similarity of the corresponding cluster set includes: For any topic field in the cluster set, the topic field embedding vector is extracted using a pre-trained word embedding model; The word embedding model is used to extract the basic field embedding vector corresponding to each basic field in the basic field set. Calculate the vector similarity between the topic field embedding vector and each basic field embedding vector to obtain the vector similarity of the corresponding basic field vector. Calculate the mean of the vector similarity of all basic field vectors to obtain the first mean of the topic field. Iterate through all topic fields in the cluster set to obtain the first mean of the corresponding topic field. The first mean of all topic fields is used to calculate the second mean, and the second mean is determined as the reference similarity of the cluster set. Traverse all cluster sets to obtain the reference similarity of the corresponding cluster sets; Obtain the field to be searched, calculate the mean similarity between the field to be searched and all basic fields, and determine the cluster set corresponding to the reference similarity closest to the mean as the search result for the field to be searched.

2. The data retrieval method according to claim 1, characterized in that, The basic field library contains N basic fields, and the historical defect data contains at least one data field, where N is an integer greater than zero; The historical defect data is filtered using a preset basic field library to obtain a set of basic fields containing at least one basic field, including: Initialize the basic field set to an empty set. For any basic field in the basic field library, match the basic field with each data field contained in the historical defect data. If the basic field matches any data field successfully, add the basic field to the basic field set. Traverse all basic fields in the basic field library to obtain the basic field set containing at least one basic field.

3. The data retrieval method according to claim 1, characterized in that, The defect description information includes at least one description field; The step of extracting topic fields from the defect description information to obtain a topic field set consisting of at least one topic field includes: For any given description field, count the number of times the description field appears in the defect description information to obtain the target number. Obtain the total number of times all description fields appear in the defect description information to obtain the total number. Calculate the first ratio between the target number and the total number, and use the first ratio as the word frequency of the description field. Obtain all reference description information contained in the preset database, count the number of reference description information containing the description field to obtain the target number, obtain the total number of all reference description information, calculate the second ratio of the total number to the target number, and use the logarithm of the second ratio as the inverse frequency of the description field. Multiply the term and the inverse frequency to determine the importance of the description field. Iterate through all description fields to obtain the importance of the corresponding description field. Description fields whose importance exceeds a preset threshold are identified as topic fields, and the topic field set is formed by all topic fields.

4. The data retrieval method according to claim 3, characterized in that, The description fields whose importance exceeds a preset threshold are identified as topic fields, and the topic field set is formed by all topic fields, including: Count the number of all description fields to obtain the field count, multiply the field count by a preset coefficient, and determine the result of the multiplication as the field identifier; Sort all description fields in descending order of importance to obtain a field sequence, with each description field corresponding to a field number in the field sequence. In the field sequence, a reference description field is determined that corresponds to the field number closest to the field identifier. The importance of the reference description field is used as the preset threshold. Description fields with an importance greater than the preset threshold are determined as topic fields. The topic field set is formed by all topic fields.

5. The data retrieval method according to any one of claims 1 to 4, characterized in that, The historical defect data includes historical defect types, and the data to be retrieved includes defect types to be retrieved. The calculation of the mean similarity between the field to be retrieved and all basic fields includes: The historical defect types and the defect types to be retrieved are compared to obtain the comparison results; If the comparison result shows that the historical defect type and the defect type to be retrieved are the same, then the mean similarity between the field to be retrieved and all basic fields in the basic field set is calculated.

6. A data retrieval device for defect analysis, characterized in that, The data retrieval device includes: The field filtering module is used to obtain historical defect data and perform basic field filtering on the historical defect data using a preset basic field library to obtain a basic field set containing at least one basic field. The field extraction module is used to obtain the defect description information of the historical defect data, extract the topic fields from the defect description information, and obtain a topic field set consisting of at least one topic field. The field comparison module is used to compare the basic field set and the topic field set, determine the intersecting fields, delete the intersecting fields from the topic field set, and obtain the target field set. The field clustering module is used to cluster all topic fields in the target field set to obtain at least one cluster set, calculate the similarity between each cluster set and the base field set, and obtain the reference similarity of the corresponding cluster set. The calculation of the similarity between each cluster set and the base field set to obtain the reference similarity of the corresponding cluster set includes: For any topic field in the cluster set, the topic field embedding vector is extracted using a pre-trained word embedding model; The word embedding model is used to extract the basic field embedding vector corresponding to each basic field in the basic field set. Calculate the vector similarity between the topic field embedding vector and each basic field embedding vector to obtain the vector similarity of the corresponding basic field vector. Calculate the mean of the vector similarity of all basic field vectors to obtain the first mean of the topic field. Iterate through all topic fields in the cluster set to obtain the first mean of the corresponding topic field. The first mean of all topic fields is used to calculate the second mean, and the second mean is determined as the reference similarity of the cluster set. Traverse all cluster sets to obtain the reference similarity of the corresponding cluster sets; The field retrieval module is used to obtain the field to be retrieved, calculate the mean similarity between the field to be retrieved and all basic fields, and determine the cluster set corresponding to the reference similarity closest to the mean as the retrieval result of the field to be retrieved.

7. A computer device, characterized in that, The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the data retrieval method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the data retrieval method as described in any one of claims 1 to 5.

Images