Image slice review method and device thereof, electronic device, and storage medium

Abstract

The application discloses a kind of image slice's review method and its device, electronic equipment and storage medium, it is related to artificial intelligence field, wherein, the review method includes: receiving image slice review request, determine the first confidence of each pseudo-marked image slice in pseudo-marked image slice set, using first identification model and second identification model respectively identify each pseudo-marked image slice in pseudo-marked image slice set, based on first mark information and two identification results, determine the preset level corresponding to each pseudo-marked image slice, using preset selection degree model filters pseudo-marked image slice in corresponding preset level, obtains the queue of image slice to be reviewed, and each image slice to be reviewed in the queue of image slice to be reviewed is reviewed.The present application solves the technical problem that the efficiency is low and the cost is high in the related art by relying only on artificial review of a large number of image slices.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and more specifically, to a method, apparatus, electronic device, and storage medium for verifying image slices. Background Technology

[0002] Currently, text labeling in the field of image text recognition can be divided into two parts: one is labeling the location information of the text in the image (i.e., forming bounding boxes), and the other is labeling the text within the bounding boxes. The process of labeling the text within the bounding boxes can be called the process of identifying information. For training image text recognition models, the localization model is often trained first using the localization information, and then the recognition model is trained using the recognition information to recognize the text within the bounding boxes. The training of the localization model and the recognition model are often separate, and the training data used for training the recognition model is pre-labeled image slice data.

[0003] Currently, much of the sample data is pseudo-labeled data. Pseudo-labeled data is characterized by having labels, but the accuracy of those labels cannot be guaranteed. In practical applications, pseudo-labeled data samples are easier to obtain than correctly labeled data samples. The advantage of pseudo-labeled data samples is that it eliminates the need for manual labeling of each image slice; instead, it requires manual verification of the pseudo-labeling results for each image slice, correcting any erroneous labels. However, training recognition models, especially large-scale models, often requires training data covering a sufficiently large number of labeled characters, and verifying the labels on this training data is a significant undertaking.

[0004] In related technologies, image slice verification methods often rely on manual verification. Even when only the labeling information of image slices needs to be verified, the verification work remains enormous when the number of image slices is large. For example, in image text recognition, as the number of training characters increases, the required image slice data grows exponentially. Current manual verification methods struggle to meet the data demands, resulting in low efficiency and high costs.

[0005] There is currently no effective solution to the above problems. Summary of the Invention

[0006] This invention provides a method, apparatus, electronic device, and storage medium for verifying image slices, to at least solve the technical problems of low efficiency and high cost in the related art that rely solely on manual verification of a large number of image slices.

[0007] According to one aspect of the present invention, a method for reviewing image slices is provided, comprising: receiving an image slice review request, wherein the image slice review request includes at least: a set of pseudo-labeled image slices; determining a first confidence level for each pseudo-labeled image slice in the set of pseudo-labeled image slices, wherein each pseudo-labeled image slice corresponds to first labeling information; identifying each pseudo-labeled image slice in the set of pseudo-labeled image slices using a first recognition model to obtain a first recognition result, and identifying each pseudo-labeled image slice in the set of pseudo-labeled image slices using a second recognition model to obtain a second recognition result, wherein the first recognition model and the second recognition model are both pre-trained recognition models; determining a preset level corresponding to each pseudo-labeled image slice based on the first labeling information, the first recognition result, and the second recognition result, wherein each preset level corresponds to a preset selection degree model, and each preset level has a different priority, with the highest preset level having the highest priority; filtering the pseudo-labeled image slices corresponding to the preset level using the preset selection degree model to obtain a queue of image slices to be reviewed, and reviewing each image slice to be reviewed in the queue of image slices to be reviewed.

[0008] Optionally, the step of determining a first confidence level for each pseudo-labeled image slice in the pseudo-labeled image slice set includes: sampling the pseudo-labeled image slice set to obtain a sample; determining the character accuracy of the pseudo-labeled image slice set based on the sample; and characterizing the character accuracy as the first confidence level for each pseudo-labeled image slice.

[0009] Optionally, the first identification result includes at least: second marker information, and the second identification result includes at least: third marker information. The step of determining a preset level corresponding to each of the pseudo-marked image slices based on the first marker information, the first identification result, and the second identification result includes: comparing the first marker information, the second marker information, and the third marker information to obtain a comparison result; determining the preset level of the pseudo-marked image slice as a first level when the comparison result indicates that the first marker information, the second marker information, and the third marker information are all consistent; determining the preset level of the pseudo-marked image slice as a second level when the comparison result indicates that any two of the first marker information, the second marker information, and the third marker information are consistent and the other marker information is inconsistent; and determining the preset level of the pseudo-marked image slice as a third level when the comparison result indicates that the first marker information, the second marker information, and the third marker information are all inconsistent.

[0010] Optionally, the first identification result further includes: a second confidence level; the second identification result further includes: a third confidence level; after determining the preset level corresponding to each pseudo-labeled image slice based on the first labeling information, the first identification result, and the second identification result, the method further includes: determining a first type of matching degree between any two of the first labeling information, the second labeling information, and the third labeling information; determining a second type of matching degree between the first labeling information, the second labeling information, and the third labeling information; establishing a preset selection degree model corresponding to the first level based on the first confidence level, the second confidence level, and the third confidence level; establishing a preset selection degree model corresponding to the second level based on the first confidence level, the second confidence level, the third confidence level, and the first type of matching degree; and establishing a preset selection degree model corresponding to the third level based on the first confidence level, the second confidence level, the third confidence level, the first type of matching degree, and the second type of matching degree.

[0011] Optionally, the step of determining a first type of matching degree between any two of the first, second, and third tag information includes: determining the total number of first characters in the first tag information, the total number of second characters in the second tag information, and the total number of third characters in the third tag information; matching the first and second tag information to obtain the number of first characters; matching the first and third tag information to obtain the number of second characters; matching the second and third tag information to obtain the number of third characters; determining a first matching degree based on the number of first characters and the total number of first characters, and determining a second matching degree based on the number of first characters and the total number of second characters; determining a third matching degree based on the number of second characters and the total number of first characters, and determining a fourth matching degree based on the number of second characters and the total number of third characters; determining a fifth matching degree based on the number of third characters and the total number of second characters, and determining a sixth matching degree based on the number of third characters and the total number of third characters; and classifying the first, second, third, fourth, fifth, and sixth matching degrees as the first type of matching degree.

[0012] Optionally, the step of determining the second type of matching degree among the first marker information, the second marker information, and the third marker information includes: matching the first marker information, the second marker information, and the third marker information to obtain the fourth character count; determining the seventh matching degree based on the fourth character count and the total first character count; determining the eighth matching degree based on the fourth character count and the total second character count; determining the ninth matching degree based on the fourth character count and the total third character count; and classifying the seventh matching degree, the eighth matching degree, and the ninth matching degree as the second type of matching degree.

[0013] Optionally, the step of using the preset selection degree model to filter the pseudo-marked image slices corresponding to the preset level to obtain a queue of image slices to be reviewed includes: using the preset selection degree model corresponding to the first level to determine a first selection degree for each pseudo-marked image slice in the first level; adding pseudo-marked images with a first selection degree less than a first preset threshold to the queue of image slices to be reviewed; using the preset selection degree model corresponding to the second level to determine a second selection degree for each pseudo-marked image slice in the second level; adding pseudo-marked images with a second selection degree less than a second preset threshold to the queue of image slices to be reviewed; using the preset selection degree model corresponding to the third level to determine a third selection degree for each pseudo-marked image slice in the third level; adding pseudo-marked images with a third selection degree less than a third preset threshold to the queue of image slices to be reviewed.

[0014] According to another aspect of the present invention, an image slice verification apparatus is also provided, comprising: a receiving unit for receiving an image slice verification request, wherein the image slice verification request includes at least: a set of pseudo-labeled image slices; a first determining unit for determining a first confidence level for each pseudo-labeled image slice in the set of pseudo-labeled image slices, wherein each pseudo-labeled image slice corresponds to first labeling information; and a recognizing unit for recognizing each pseudo-labeled image slice in the set of pseudo-labeled image slices using a first recognizing model to obtain a first recognizing result, and recognizing each pseudo-labeled image slice in the set of pseudo-labeled image slices using a second recognizing model to obtain a first recognizing result. The second recognition result is obtained, wherein the first recognition model and the second recognition model are both pre-trained recognition models; the second determining unit is used to determine the preset level corresponding to each pseudo-labeled image slice based on the first label information, the first recognition result and the second recognition result, wherein each preset level corresponds to a preset selection degree model, and each preset level has a different priority, with the highest preset level having the highest priority; the filtering unit is used to filter the pseudo-labeled image slices in the corresponding preset level using the preset selection degree model to obtain a queue of image slices to be reviewed, and to review each image slice to be reviewed in the queue of image slices to be reviewed.

[0015] Optionally, the first determining unit includes: a first sampling module, used to sample the set of pseudo-labeled image slices to obtain a sample; a first determining module, used to determine the character accuracy of the set of pseudo-labeled image slices based on the sample; and a first characterization module, used to characterize the character accuracy as the first confidence level of each pseudo-labeled image slice.

[0016] Optionally, the first identification result includes at least: second marker information, the second identification result includes at least: third marker information, and the second determining unit includes: a first comparison module, configured to compare the first marker information, the second marker information, and the third marker information to obtain a comparison result; a second determining module, configured to determine the preset level of the pseudo-marked image slice as a first level when the comparison result indicates that the first marker information, the second marker information, and the third marker information are all consistent; a third determining module, configured to determine the preset level of the pseudo-marked image slice as a second level when the comparison result indicates that any two of the first marker information, the second marker information, and the third marker information are consistent and the other marker information is inconsistent; and a fourth determining module, configured to determine the preset level of the pseudo-marked image slice as a third level when the comparison result indicates that the first marker information, the second marker information, and the third marker information are all inconsistent.

[0017] Optionally, the first identification result further includes a second confidence level, and the second identification result further includes a third confidence level. The verification device further includes a fifth determining module, configured to determine a first type of matching degree between any two of the first, second, and third labeling information after determining a preset level corresponding to each pseudo-labeled image slice based on the first labeling information, the first identification result, and the second identification result; a sixth determining module, configured to determine a second type of matching degree between the first, second, and third labeling information; a first establishing module, configured to establish a preset selection degree model corresponding to the first level based on the first confidence level, the second confidence level, and the third confidence level; a second establishing module, configured to establish a preset selection degree model corresponding to the second level based on the first confidence level, the second confidence level, the third confidence level, and the first type of matching degree; and a third establishing module, configured to establish a preset selection degree model corresponding to the third level based on the first confidence level, the second confidence level, the third confidence level, the first type of matching degree, and the second type of matching degree.

[0018] Optionally, the fifth determining module includes: a first determining submodule, configured to determine the total number of first characters in the first marker information, the total number of second characters in the second marker information, and the total number of third characters in the third marker information, respectively; a first matching submodule, configured to match the first marker information and the second marker information to obtain the number of first characters; a second matching submodule, configured to match the first marker information and the third marker information to obtain the number of second characters; a third matching submodule, configured to match the second marker information and the third marker information to obtain the number of third characters; and a second determining submodule, configured to determine a first matching degree based on the number of first characters and the total number of first characters, and based on... The first character count and the total second character count are used to determine a second matching degree; a third determining submodule is used to determine a third matching degree based on the second character count and the total first character count, and a fourth matching degree based on the second character count and the total third character count; a fourth determining submodule is used to determine a fifth matching degree based on the third character count and the total second character count, and a sixth matching degree based on the third character count and the total third character count; a first classifying submodule is used to classify the first matching degree, the second matching degree, the third matching degree, the fourth matching degree, the fifth matching degree, and the sixth matching degree into the first type of matching degree.

[0019] Optionally, the sixth determining module includes: a fourth matching submodule, used to match the first marker information, the second marker information, and the third marker information to obtain a fourth character count; a fifth determining submodule, used to determine a seventh matching degree based on the fourth character count and the total number of the first characters; a sixth determining submodule, used to determine an eighth matching degree based on the fourth character count and the total number of the second characters; a seventh determining submodule, used to determine a ninth matching degree based on the fourth character count and the total number of the third characters; and a second classification submodule, used to classify the seventh matching degree, the eighth matching degree, and the ninth matching degree into the second type of matching degree.

[0020] Optionally, the filtering unit includes: a seventh determining module, configured to determine a first selection degree for each pseudo-marked image slice in the first level using the preset selection degree model corresponding to the first level; a first placing module, configured to place pseudo-marked images with a first selection degree less than a first preset threshold into the image slice queue to be reviewed; an eighth determining module, configured to determine a second selection degree for each pseudo-marked image slice in the second level using the preset selection degree model corresponding to the second level; a second placing module, configured to place pseudo-marked images with a second selection degree less than a second preset threshold into the image slice queue to be reviewed; a ninth determining module, configured to determine a third selection degree for each pseudo-marked image slice in the third level using the preset selection degree model corresponding to the third level; and a third placing module, configured to place pseudo-marked images with a third selection degree less than a third preset threshold into the image slice queue to be reviewed.

[0021] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the above-described image slice verification method.

[0022] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the above-described image slicing verification method.

[0023] In this disclosure, an image slice verification request is received, a first confidence level of each pseudo-labeled image slice in the pseudo-labeled image slice set is determined, each pseudo-labeled image slice in the pseudo-labeled image slice set is identified using a first recognition model to obtain a first recognition result, and each pseudo-labeled image slice in the pseudo-labeled image slice set is identified using a second recognition model to obtain a second recognition result. Based on the first labeling information, the first recognition result, and the second recognition result, a preset level corresponding to each pseudo-labeled image slice is determined, and pseudo-labeled image slices in the corresponding preset level are filtered using a preset selection degree model to obtain a queue of image slices to be verified, and each image slice to be verified in the queue of image slices to be verified is verified. In this disclosure, a first confidence level for each pseudo-labeled image slice in the pseudo-labeled image slice set can be determined first, and different recognition models can be used to identify each pseudo-labeled image slice in the pseudo-labeled image slice set respectively. Then, based on the first labeling information of each pseudo-labeled image slice and the different recognition results, a preset level corresponding to the pseudo-labeled image slice is determined. Then, a preset selection degree model is used to filter the pseudo-labeled image slices in the corresponding preset level, and the labeling information of the filtered pseudo-labeled image slices is reviewed. This can reduce the number of image slices that need to be reviewed, effectively improve the review efficiency, reduce the cost, and thus solve the technical problem in related technologies where relying solely on manual review of a large number of image slices is inefficient and costly. Attached Figure Description

[0024] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0025] Figure 1 This is a flowchart of an optional image slicing verification method according to an embodiment of the present invention;

[0026] Figure 2 This is a flowchart of an optional method for determining a preset level of a pseudo-marked image slice according to an embodiment of the present invention;

[0027] Figure 3 This is a schematic diagram of an optional verification pseudo-marker image slicing process according to an embodiment of the present invention;

[0028] Figure 4 This is a schematic diagram of an optional image slice verification device according to an embodiment of the present invention;

[0029] Figure 5 This is a hardware structure block diagram of an electronic device (or mobile device) for a verification method of image slicing according to an embodiment of the present invention. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] To facilitate understanding of the present invention by those skilled in the art, some terms or nouns involved in the various embodiments of the present invention are explained below:

[0033] Confidence level: A quantitative value used to indicate the likelihood that a model believes its output is correct.

[0034] Pseudo-labeled samples: These are samples that have labels, but the correctness of the labels cannot be guaranteed.

[0035] Image slicing: In image text recognition, the portion of the image containing text is sliced ​​out based on the text location box.

[0036] It should be noted that the image slice verification method and apparatus in this disclosure can be used in the field of artificial intelligence for image slice verification, and can also be used in any field other than artificial intelligence for image slice verification. This disclosure does not limit the application field of the image slice verification method and apparatus.

[0037] It should be noted that all information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this disclosure are information and data authorized by the user or fully authorized by all parties. For example, this system has an interface with relevant users or organizations. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain the relevant information after receiving consent information from the aforementioned user or organization.

[0038] The following embodiments of the present invention can be applied to various systems / applications / devices for verifying image slices. The present invention assigns pseudo-confidence to pseudo-labeled image slices and, by combining the confidence scores output by different recognition models with labeling information, can classify pseudo-labeled image slices into different levels. Then, a selection degree calculation formula (conX*tan((Π / 2)*Y, where conX represents a certain confidence score and Y represents a certain matching degree) is constructed for each level. This selection degree calculation formula, by fusing confidence and matching degree information, can filter out error-prone samples in pseudo-labeled image slices. These samples are then subject to focused manual verification, reducing the complex and labor-intensive work involved in verifying image slice data and thus solving the problem of high costs in verifying pseudo-labeled samples in the field of image text recognition.

[0039] The present invention will now be described in detail with reference to various embodiments.

[0040] Example 1

[0041] According to an embodiment of the present invention, an embodiment of an image slice verification method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0042] Figure 1 This is a flowchart of an optional image slicing verification method according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0043] Step S101: Receive an image slice verification request, wherein the image slice verification request includes at least: a set of pseudo-labeled image slices.

[0044] Step S102: Determine the first confidence level of each pseudo-labeled image slice in the pseudo-labeled image slice set, wherein each pseudo-labeled image slice corresponds to first label information.

[0045] Step S103: Use the first recognition model to identify each pseudo-labeled image slice in the pseudo-labeled image slice set to obtain the first recognition result, and use the second recognition model to identify each pseudo-labeled image slice in the pseudo-labeled image slice set to obtain the second recognition result. The first recognition model and the second recognition model are both pre-trained recognition models.

[0046] Step S104: Based on the first marker information, the first recognition result and the second recognition result, determine the preset level corresponding to each pseudo-marked image slice. Each preset level corresponds to a preset selection degree model. Each preset level has a different priority, with the highest preset level having the highest priority.

[0047] Step S105: Use a preset selection degree model to filter the pseudo-marked image slices in the corresponding preset level to obtain a queue of image slices to be reviewed, and review each image slice in the queue of image slices to be reviewed.

[0048] Through the above steps, an image slice verification request can be received, a first confidence level of each pseudo-labeled image slice in the pseudo-labeled image slice set can be determined, a first recognition model can be used to identify each pseudo-labeled image slice in the pseudo-labeled image slice set to obtain a first recognition result, and a second recognition model can be used to identify each pseudo-labeled image slice in the pseudo-labeled image slice set to obtain a second recognition result. Based on the first labeling information, the first recognition result, and the second recognition result, a preset level corresponding to each pseudo-labeled image slice can be determined, and a preset selection degree model can be used to filter the pseudo-labeled image slices in the corresponding preset level to obtain a queue of image slices to be verified, and each image slice to be verified in the queue of image slices to be verified can be verified. In this embodiment of the invention, the first confidence level of each pseudo-labeled image slice in the pseudo-labeled image slice set can be determined first, and each pseudo-labeled image slice in the pseudo-labeled image slice set can be identified by different recognition models. Then, based on the first labeling information of each pseudo-labeled image slice and the different recognition results, the preset level corresponding to the pseudo-labeled image slice can be determined. Then, the pseudo-labeled image slices in the corresponding preset level can be filtered by a preset selection degree model, and the labeling information of the filtered pseudo-labeled image slices can be reviewed. This can reduce the number of image slices that need to be reviewed, effectively improve the review efficiency, reduce the cost, and thus solve the technical problem of low efficiency and high cost of relying solely on manual review of a large number of image slices in related technologies.

[0049] The embodiments of the present invention will now be described in detail with reference to the steps described above.

[0050] Step S101: Receive an image slice verification request, wherein the image slice verification request includes at least: a set of pseudo-labeled image slices.

[0051] In this embodiment of the invention, an image slice verification request may be received first. The image slice verification request carries a set of pseudo-labeled image slices. Each pseudo-labeled image slice in the set of pseudo-labeled image slices is an image slice that has been labeled, but the correctness of the label cannot be guaranteed.

[0052] Step S102: Determine the first confidence level of each pseudo-labeled image slice in the pseudo-labeled image slice set, wherein each pseudo-labeled image slice corresponds to first label information.

[0053] Optionally, the step of determining a first confidence level for each pseudo-labeled image slice in the pseudo-labeled image slice set includes: sampling the pseudo-labeled image slice set to obtain a sample; determining the character accuracy of the pseudo-labeled image slice set based on the sample; and characterizing the character accuracy as a first confidence level for each pseudo-labeled image slice.

[0054] In this embodiment of the invention, a confidence level can be determined for the set of pseudo-labeled image slices. Specifically, random sampling can be performed on the set of pseudo-labeled image slices to obtain sampled samples. Then, the random sampled samples are checked to obtain the character accuracy of the set of pseudo-labeled image slices (i.e., the character accuracy of the set of pseudo-labeled image slices is determined based on the sampled samples). The obtained character accuracy is used as the confidence level conFa of the set of pseudo-labeled image slices. This confidence level is a pseudo-confidence level and can be used as the confidence level of each pseudo-labeled image slice in the set of pseudo-labeled image slices (i.e., the character accuracy is represented as the first confidence level of each pseudo-labeled image slice). Furthermore, each pseudo-labeled image slice corresponds to first label information (label information).

[0055] Step S103: Use the first recognition model to identify each pseudo-labeled image slice in the pseudo-labeled image slice set to obtain the first recognition result, and use the second recognition model to identify each pseudo-labeled image slice in the pseudo-labeled image slice set to obtain the second recognition result. The first recognition model and the second recognition model are both pre-trained recognition models.

[0056] In this embodiment of the invention, two pre-trained recognition models (i.e., a first recognition model and a second recognition model, both of which are pre-trained recognition models) can be used to recognize each pseudo-labeled image slice in the set of pseudo-labeled image slices, and a first recognition result and a second recognition result can be obtained respectively. The first recognition result includes: a second confidence level conA (i.e., the probability of correct recognition) and a second labeling information (including the recognized characters and the number of characters) for each pseudo-labeled image slice. The second recognition result includes: a third confidence level conB (i.e., the probability of correct recognition) and a third labeling information (including the recognized characters and the number of characters) for each pseudo-labeled image slice.

[0057] Step S104: Based on the first marker information, the first recognition result and the second recognition result, determine the preset level corresponding to each pseudo-marked image slice. Each preset level corresponds to a preset selection degree model. Each preset level has a different priority, with the highest preset level having the highest priority.

[0058] Optionally, the first identification result includes at least: second labeling information, and the second identification result includes at least: third labeling information. The step of determining the preset level corresponding to each pseudo-labeled image slice based on the first labeling information, the first identification result, and the second identification result includes: comparing the first labeling information, the second labeling information, and the third labeling information to obtain a comparison result; determining the preset level of the pseudo-labeled image slice as the first level when the comparison result indicates that the first labeling information, the second labeling information, and the third labeling information are all consistent; determining the preset level of the pseudo-labeled image slice as the second level when the comparison result indicates that any two of the first labeling information, the second labeling information, and the third labeling information are consistent and the other labeling information is inconsistent; and determining the preset level of the pseudo-labeled image slice as the third level when the comparison result indicates that the first labeling information, the second labeling information, and the third labeling information are all inconsistent.

[0059] In this embodiment of the invention, a preset level corresponding to each pseudo-marked image slice can be determined based on the matching situation between the first marking information, the first recognition result and the second recognition result. Furthermore, each preset level corresponds to a preset selection degree model, and each preset level has a different priority, with the highest preset level having the highest priority (i.e., the pseudo-marked image slice corresponding to the highest preset level is checked first). Specifically: First, the first, second, and third marker information can be compared to obtain the comparison result. If the comparison result indicates that the first, second, and third marker information are all consistent (i.e., the marker information recognized by both recognition models is consistent with the pseudo-marker information), then the preset level of the pseudo-marker image slice is determined to be the first level. If the comparison result indicates that any two of the first, second, and third marker information are consistent and the other is inconsistent (i.e., any two of the marker information recognized by the two recognition models and the pseudo-marker information are consistent, but the other is inconsistent), then the preset level of the pseudo-marker image slice is determined to be the second level. If the comparison result indicates that the first, second, and third marker information are all inconsistent (i.e., the marker information recognized by the two recognition models and the pseudo-marker information are all inconsistent), then the preset level of the pseudo-marker image slice is determined to be the third level.

[0060] Figure 2This is a flowchart illustrating an optional method for determining a preset level of a pseudo-marked image slice according to an embodiment of the present invention, such as... Figure 2 As shown, it includes the following steps:

[0061] Step S201: Compare the first marker information, the second marker information, and the third marker information to obtain the comparison result;

[0062] Step S202: If the comparison results indicate that the first marker information, the second marker information, and the third marker information are all consistent, the preset level of the pseudo-marked image slice is determined to be the first level.

[0063] Step S203: If the comparison result indicates that any two of the first, second, and third marker information are consistent and the third marker information is inconsistent, the preset level of the pseudo-marked image slice is determined to be the second level.

[0064] Step S204: If the comparison results indicate that the first marker information, the second marker information, and the third marker information are inconsistent, the preset level of the pseudo-marked image slice is determined to be the third level.

[0065] Optionally, the first identification result further includes: a second confidence level; the second identification result further includes: a third confidence level; after determining the preset level corresponding to each pseudo-labeled image slice based on the first label information, the first identification result, and the second identification result, the method further includes: determining a first type of matching degree between any two labels among the first label information, the second label information, and the third label information; determining a second type of matching degree between the first label information, the second label information, and the third label information; establishing a preset selection degree model corresponding to the first level based on the first confidence level, the second confidence level, and the third confidence level; establishing a preset selection degree model corresponding to the second level based on the first confidence level, the second confidence level, the third confidence level, and the first type of matching degree; and establishing a preset selection degree model corresponding to the third level based on the first confidence level, the second confidence level, the third confidence level, the first type of matching degree, and the second type of matching degree.

[0066] In this embodiment of the invention, a first type of matching degree can be determined between any two of the first, second, and third tag information, and a second type of matching degree can be determined between the first, second, and third tag information. The matching degree is used to measure the consistency between the tag information, such as the consistency between the first and second tag information, or the consistency between the first, second, and third tag information.

[0067] For the first level, each sample (i.e., the pseudo-labeled image slice) has three confidence levels: the confidence level conA recognized by the A recognition model (i.e., the first recognition model), the confidence level conB recognized by the B recognition model (i.e., the second recognition model), and the determination pseudo-confidence level conFa. These three confidence levels (i.e., the first confidence level, the second confidence level, and the third confidence level) can be accumulated to establish a preset selection degree model (i.e., the selection degree calculation formula) corresponding to the first level. The selection degree calculation formula Se for this level is: Se = conA + conB + conFa.

[0068] For the second level, there are three cases: (1) the label information identified by the A identification model is consistent with the pseudo-label information, but inconsistent with the label information identified by the B identification model; (2) the label information identified by the A identification model is inconsistent with the pseudo-label information, but consistent with the label information identified by the B identification model; (3) the label information identified by the B identification model is inconsistent with the pseudo-label information, but consistent with the label information identified by the A identification model. For these three different cases, a preset selection degree model corresponding to the second level can be established based on the first confidence level, the second confidence level, the third confidence level, and the first type of matching degree, as follows:

[0069] In case (1), calculate the matching degree M between Π / 2 and the first type. A(AB) The first product between (Π / 2)*M A(AB) Using the tan function, calculate the first function value of the first product: tan((Π / 2)*M A(AB) ), calculate the second product between the second confidence level conA and the first function value, conAtan((Π / 2)*M A(AB) ), calculate the matching degree M between Π / 2 and the first type. B(AB) The third product between (Π / 2)M B(AB) Using the tan function, calculate the second function value of the third product: tan((Π / 2)M B(AB) ), calculate the fourth product between the third confidence level conB and the second function value, conAtan((Π / 2)*M A(AB) By summing the second and fourth products, we obtain the formula for calculating the selectivity:

[0070] Se=conAtan((Π / 2)*M A(AB) )+conBtan((Π / 2)M B(AB) );

[0071] In case (2), calculate the matching degree M between Π / 2 and the first type. A(AFa) The fifth product between (Π / 2)*M A(AFa) Using the tan function, calculate the third function value of the fifth product: tan((Π / 2)*M A(AFa)), calculate the sixth product between the second confidence level conA and the third function value, conAtan((Π / 2)*M A(AFa) ), calculate the matching degree M between Π / 2 and the first type. Fa(AFa) The seventh product between (Π / 2)M Fa(AFa) Using the tan function, calculate the fourth function value of the seventh product: tan((Π / 2)M Fa(AFa) ), calculate the eighth product between the first confidence level conFa and the fourth function value, conFatan((Π / 2)M Fa(AFa) Adding the sixth and eighth products together, we get the selection degree calculation formula:

[0072] Se=conAtan((Π / 2)*M A(AFa) )+conFatan((Π / 2)M Fa(AFa) );

[0073] In case (3), calculate the matching degree M between Π / 2 and the first type. B(BFa) The ninth product between (Π / 2)*M B(BFa) Using the tan function, calculate the fifth function value of the ninth product: tan((Π / 2)*M B(BFa) ), calculate the tenth product between the third confidence level conB and the fifth function value, conBtan((Π / 2)*M B(BFa) ), calculate the matching degree M between Π / 2 and the first type. Fa(BFa) The eleventh product between (Π / 2)M Fa(BFa) Using the tan function, calculate the sixth function value of the eleventh product: tan((Π / 2)M Fa(BFa) ), calculate the twelfth product between the first confidence level conFa and the sixth function value conFatan((Π / 2)M). Fa(BFa) Adding the tenth and twelfth products together, we get the selection degree calculation formula:

[0074] Se=conBtan((Π / 2)*M B(BFa) )+conFatan((Π / 2)M Fa(BFa) );

[0075] Where M represents the matching degree, which is the number of characters with the same tag information divided by the total number of characters in the corresponding tag information. A(AB) M represents the number of characters whose tag information is consistent when the two recognition models A and B are identified, divided by the total number of characters in the tag information identified by recognition model A. B(AB) M represents the number of characters whose tag information is consistent when the two recognition models A and B are identified, divided by the total number of characters in the tag information identified by the B recognition model. A(AFa)M represents the number of characters whose marked information and pseudo-mark information match as identified by the A recognition model / the total number of marked information characters identified by the A recognition model. Fa(AFa) M represents the number of characters in the A recognition model that match the pseudo-label information, divided by the total number of pseudo-label characters. B(BFa) M represents the number of characters whose marked information and pseudo-mark information match as identified by the B recognition model, divided by the total number of marked information characters identified by the B recognition model. Fa(BFa) This represents the number of characters in the B recognition model that match the pseudo-mark information, divided by the total number of pseudo-mark information characters.

[0076] For the third level, a preset selection factor model corresponding to the third level can be established based on the first confidence level, second confidence level, third confidence level, first-type matching degree, and second-type matching degree. Specifically, the product between Π / 2 and the first-type matching degree, and the product between Π / 2 and the second-type matching degree can be calculated separately. The tan function is used to calculate the function value of each product. Each function value is multiplied by any confidence level of the corresponding matching tag information to obtain multiple product values. All product values ​​are summed to obtain the selection factor calculation formula Se for this level:

[0077] Se=conAtan((Π / 2)*M A(AB) )+conBtan((Π / 2)M B(AB) )+

[0078] conAtan((Π / 2)*M A(AFa) )+conFatan((Π / 2)M Fa(AFa) )+

[0079] conBtan((Π / 2)*M B(BFa) )+conFatan((Π / 2)M Fa(BFa) )+;

[0080] conA(tan(Π / 2)*M A(ABFa) )+conBtan((Π / 2)*M B(ABFa) )+

[0081] conFa((Π / 2)*M Fa(ABFa) )

[0082] Among them, M A(ABFa) M represents the number of characters whose marker information and pseudo-mark information are consistent, divided by the total number of marker information characters recognized by the A / B recognition model. B(ABFa) M represents the number of characters whose marker information and pseudo-mark information are consistent, divided by the total number of marker information characters recognized by the B recognition model. Fa(ABFa)This represents the number of characters whose identified marker information and pseudo-mark information are consistent, divided by the total number of pseudo-mark information characters.

[0083] In this embodiment of the invention, the selection factor calculation formula conX*tan((Π / 2)*Y, by fusing confidence and matching information, can filter out samples prone to errors in pseudo-labeled image slices, and these samples are then handed over to manual review. Furthermore, the tan function does not need to be set to a very small value because for each level and each case, cases with a matching factor of 1 have already been excluded. For the confidence factor, if the confidence factor is 0, it will be selected first, so there is no need to consider cases where the tan factor is 0, nor is it necessary to set a very small value to prevent the tan factor from becoming infinite. In addition, the tan function has good discrimination between 0 and 1. For manual review, the label accuracy is generally high, and the sample matching degree is generally also high, requiring a function that can better distinguish manually reviewed samples. In this case, choosing the tan function as the function in the selection factor calculation formula is more appropriate.

[0084] Optionally, the step of determining the first type of matching degree between any two of the first, second, and third tag information includes: determining the total number of first characters in the first tag information, the total number of second characters in the second tag information, and the total number of third characters in the third tag information, respectively; matching the first and second tag information to obtain the number of first characters; matching the first and third tag information to obtain the number of second characters; matching the second and third tag information to obtain the number of third characters; determining the first matching degree based on the number of first characters and the total number of first characters, and determining the second matching degree based on the number of first characters and the total number of second characters; determining the third matching degree based on the number of second characters and the total number of first characters, and determining the fourth matching degree based on the number of second characters and the total number of third characters; determining the fifth matching degree based on the number of third characters and the total number of second characters, and determining the sixth matching degree based on the number of third characters and the total number of third characters; and classifying the first, second, third, fourth, fifth, and sixth matching degrees as the first type of matching degree.

[0085] In this embodiment of the invention, a first-class matching degree between any two of the first, second, and third marker information can be determined first. Specifically, the total number of first characters in the first marker information, the total number of second characters in the second marker information, and the total number of third characters in the third marker information can be determined respectively. Then, the first and second marker information are matched to obtain the number of first characters; the first and third marker information are matched to obtain the number of second characters; and the second and third marker information are matched to obtain the number of third characters. Afterward, the matching degree is obtained based on the ratio of the number of matching identical characters to the total number of characters in each marker information. For example, the first matching degree is determined based on the ratio of the number of first characters to the total number of first characters; the second matching degree is determined based on the ratio of the number of first characters to the total number of second characters; the third matching degree is determined based on the ratio of the number of second characters to the total number of first characters; the fourth matching degree is determined based on the ratio of the number of second characters to the total number of third characters; the fifth matching degree is determined based on the ratio of the number of third characters to the total number of second characters; and the sixth matching degree is determined based on the ratio of the number of third characters to the total number of third characters. The first, second, third, fourth, fifth, and sixth matching degrees are then categorized into the first type of matching degree.

[0086] Optionally, the step of determining the second type of matching degree among the first tag information, the second tag information, and the third tag information includes: matching the first tag information, the second tag information, and the third tag information to obtain the fourth character count; determining the seventh matching degree based on the fourth character count and the total first character count; determining the eighth matching degree based on the fourth character count and the total second character count; determining the ninth matching degree based on the fourth character count and the total third character count; and classifying the seventh matching degree, the eighth matching degree, and the ninth matching degree as the second type of matching degree.

[0087] In this embodiment of the invention, a second type of matching degree can be determined first among the first tag information, the second tag information, and the third tag information. Specifically, the first tag information, the second tag information, and the third tag information can be matched first to obtain the number of fourth characters. Then, the seventh matching degree is determined based on the ratio of the number of fourth characters to the total number of first characters. The eighth matching degree is determined based on the ratio of the number of fourth characters to the total number of second characters. The ninth matching degree is determined based on the ratio of the number of fourth characters to the total number of third characters. The seventh matching degree, the eighth matching degree, and the ninth matching degree are then classified as the second type of matching degree.

[0088] Step S105: Use a preset selection degree model to filter the pseudo-marked image slices in the corresponding preset level to obtain a queue of image slices to be reviewed, and review each image slice in the queue of image slices to be reviewed.

[0089] Optionally, the step of using a preset selection degree model to filter pseudo-labeled image slices in a corresponding preset level to obtain a queue of image slices to be reviewed includes: using a preset selection degree model corresponding to the first level to determine the first selection degree of each pseudo-labeled image slice in the first level; adding pseudo-labeled images with a first selection degree less than a first preset threshold to the queue of image slices to be reviewed; using a preset selection degree model corresponding to the second level to determine the second selection degree of each pseudo-labeled image slice in the second level; adding pseudo-labeled images with a second selection degree less than a second preset threshold to the queue of image slices to be reviewed; using a preset selection degree model corresponding to the third level to determine the third selection degree of each pseudo-labeled image slice in the third level; adding pseudo-labeled images with a third selection degree less than a third preset threshold to the queue of image slices to be reviewed.

[0090] In this embodiment of the invention, a preset selection degree model can be used to filter pseudo-labeled image slices in the corresponding preset level to obtain a queue of image slices to be reviewed. Then, each image slice in the queue of image slices to be reviewed is reviewed. Specifically, according to the different priorities between different levels, the highest level (i.e., the third level) is determined to have the highest priority, and the lowest level (i.e., the first level) has the lowest priority. That is, the priority of the third level is greater than that of the second level, which is greater than that of the first level. Moreover, the priority of the level is higher than that of the sorting. Regardless of whether the selection degree obtained by the previous level is greater than that of the next level, the priority of the level is higher than that of the sorting. This allows for the sorting of pseudo-labeled image slices according to their hierarchy. Pseudo-labeled image slices with higher hierarchy priority are reviewed first (i.e., a preset selection degree model corresponding to the first hierarchy is used to determine the first selection degree of each pseudo-labeled image slice in the first hierarchy, and pseudo-labeled image slices with a first selection degree less than a first preset threshold are placed in the image slice queue to be reviewed; a preset selection degree model corresponding to the second hierarchy is used to determine the second selection degree of each pseudo-labeled image slice in the second hierarchy, and pseudo-labeled image slices with a second selection degree less than a second preset threshold are placed in the image slice queue to be reviewed; a preset selection degree model corresponding to the third hierarchy is used to determine the third selection degree of each pseudo-labeled image slice in the third hierarchy, and pseudo-labeled image slices with a third selection degree less than a third preset threshold are placed in the image slice queue to be reviewed). For example, pseudo-labeled image slices with low selection degree at higher hierarchy levels are reviewed first because these samples are prone to errors and are most likely to be mislabeled. Pseudo-labeled image slices with high selection degree at lower hierarchy levels do not need to be labeled because these samples are generally correctly labeled.

[0091] In this embodiment of the invention, after reviewing the selected image slices that require verification, if the verification result indicates that the annotation information on the image slices is incorrect, it can be modified. Then, the recognition model is trained using image slices with modified annotation information, image slices with correctly verified annotation information, and image slices that do not require verification. Alternatively, the recognition model can be trained first using image slices that do not require verification, and then further trained using verified image slices.

[0092] The following describes in detail another optional implementation method.

[0093] Figure 3 This is a schematic diagram of an optional verification pseudo-marker image slicing process according to an embodiment of the present invention, such as... Figure 3 As shown, the process is as follows:

[0094] (1) Input pseudo-labeled image slices and two trained recognition models A and B.

[0095] (2) Determine the confidence of the pseudo-labeled image slice: By random sampling, the sample obtained by random sampling is checked to obtain the character accuracy of the pseudo-labeled data sample, and the obtained character accuracy is used as the confidence of the pseudo-labeled image slice, conFa. Thus, the pseudo-labeled image slice has pseudo-label information RecFa and pseudo confidence ConFa.

[0096] (3) The pseudo-labeled image slices are identified using recognition models A and B to obtain the label information RecA and RecB, and the confidence levels of the label information conA and conB.

[0097] (4) Determine whether the pseudo-mark information RecFa, the mark information RecA and RecB are consistent.

[0098] (5) If the three label information are consistent, they are divided into the first level, and then sorted by the corresponding selection degree calculation formula. The samples are sorted from small to large by the obtained selection degree value.

[0099] (6) If two labels are consistent and the third label is inconsistent, they are divided into the second level. Then, the corresponding selection degree calculation formula is used to sort them, and the samples are sorted from small to large by the obtained selection degree values.

[0100] (7) If the three label information are inconsistent, they are divided into the third level, and then sorted by the corresponding selection degree calculation formula, and the samples are sorted from small to large by the obtained selection degree value.

[0101] (8) Select samples that need to be reviewed according to the hierarchy. Samples with low selection degree at higher levels should be reviewed first. The hierarchy has higher priority than selection degree. Samples with high selection degree at lower levels do not need to be reviewed.

[0102] (9) Review the selected samples and adjust the labeling content.

[0103] In this embodiment of the invention, for image slice label verification, image slices most likely to be false labels can be selected and handed over to manual verification, while image slices with a low probability of being false labels do not need to be verified. This achieves the goal of not verifying all image slices one by one, greatly reducing the number of verifications. Furthermore, using model recognition results and false label information for grading makes the method for reducing verification workload more flexible. For example, for the first level, human experience can be used to select samples that are not verified or have low selection scores. If the data within the first level is not verified, then it is not necessary to wait for the model to produce all recognition results before determining which samples to verify and which not to verify based on selection scores, because samples within the first level are not verified. Moreover, in some scenarios, only grading can be performed without calculating selection scores to select samples that need verification. In addition, sorting by selection score allows the selected false-labeled samples to be adjusted first and used for training, or for further grading. It is also possible to identify and filter simultaneously, with samples with low selection scores handed over to manual verification. This will improve the review process, reduce manual workload, increase work efficiency, and drive the review process towards being more time-saving, labor-saving, more precise, more efficient, and less costly.

[0104] The following is a detailed description with reference to another embodiment.

[0105] Example 2

[0106] The image slicing verification device provided in this embodiment includes multiple implementation units, each of which corresponds to a specific implementation step in Embodiment 1 above.

[0107] Figure 4 This is a schematic diagram of an optional image slice verification device according to an embodiment of the present invention, such as... Figure 4 As shown, the verification device may include: a receiving unit 40, a first determining unit 41, an identification unit 42, a second determining unit 43, and a filtering unit 44, wherein...

[0108] The receiving unit 40 is configured to receive an image slice verification request, wherein the image slice verification request includes at least: a set of pseudo-marked image slices;

[0109] The first determining unit 41 is used to determine the first confidence level of each pseudo-labeled image slice in the pseudo-labeled image slice set, wherein each pseudo-labeled image slice corresponds to first labeling information;

[0110] The recognition unit 42 is used to recognize each pseudo-labeled image slice in the pseudo-labeled image slice set using a first recognition model to obtain a first recognition result, and to recognize each pseudo-labeled image slice in the pseudo-labeled image slice set using a second recognition model to obtain a second recognition result, wherein the first recognition model and the second recognition model are both pre-trained recognition models;

[0111] The second determining unit 43 is used to determine the preset level corresponding to each pseudo-marked image slice based on the first marking information, the first recognition result and the second recognition result. Each preset level corresponds to a preset selection degree model, and each preset level has a different priority, with the highest preset level having the highest priority.

[0112] The filtering unit 44 is used to filter the pseudo-marked image slices in the corresponding preset level using a preset selection degree model to obtain a queue of image slices to be reviewed, and to review each image slice to be reviewed in the queue of image slices to be reviewed.

[0113] The aforementioned verification device can receive image slice verification requests through receiving unit 40, determine the first confidence level of each pseudo-labeled image slice in the pseudo-labeled image slice set through first determining unit 41, identify each pseudo-labeled image slice in the pseudo-labeled image slice set through identification unit 42 using a first identification model to obtain a first identification result, and identify each pseudo-labeled image slice in the pseudo-labeled image slice set through a second identification model to obtain a second identification result, determine the preset level corresponding to each pseudo-labeled image slice through second determining unit 43 based on the first labeling information, the first identification result, and the second identification result, and filter the pseudo-labeled image slices in the corresponding preset level through filtering unit 44 using a preset selection degree model to obtain a queue of image slices to be verified, and verify each image slice to be verified in the queue of image slices to be verified. In this embodiment of the invention, the first confidence level of each pseudo-labeled image slice in the pseudo-labeled image slice set can be determined first, and each pseudo-labeled image slice in the pseudo-labeled image slice set can be identified by different recognition models. Then, based on the first labeling information of each pseudo-labeled image slice and the different recognition results, the preset level corresponding to the pseudo-labeled image slice can be determined. Then, the pseudo-labeled image slices in the corresponding preset level can be filtered by a preset selection degree model, and the labeling information of the filtered pseudo-labeled image slices can be reviewed. This can reduce the number of image slices that need to be reviewed, effectively improve the review efficiency, reduce the cost, and thus solve the technical problem of low efficiency and high cost of relying solely on manual review of a large number of image slices in related technologies.

[0114] Optionally, the first determining unit includes: a first sampling module for sampling the set of pseudo-labeled image slices to obtain sampled samples; a first determining module for determining the character accuracy of the set of pseudo-labeled image slices based on the sampled samples; and a first characterization module for characterizing the character accuracy as a first confidence level for each pseudo-labeled image slice.

[0115] Optionally, the first identification result includes at least: second labeling information, the second identification result includes at least: third labeling information, and the second determining unit includes: a first comparison module, used to compare the first labeling information, the second labeling information, and the third labeling information to obtain a comparison result; a second determining module, used to determine the preset level of the pseudo-labeled image slice as the first level when the comparison result indicates that the first labeling information, the second labeling information, and the third labeling information are all consistent; a third determining module, used to determine the preset level of the pseudo-labeled image slice as the second level when the comparison result indicates that any two of the first labeling information, the second labeling information, and the third labeling information are consistent and the other labeling information is inconsistent; and a fourth determining module, used to determine the preset level of the pseudo-labeled image slice as the third level when the comparison result indicates that the first labeling information, the second labeling information, and the third labeling information are all inconsistent.

[0116] Optionally, the first identification result further includes: a second confidence level; the second identification result further includes: a third confidence level; the verification device further includes: a fifth determining module, used to determine a first type of matching degree between any two of the first, second, and third labeling information after determining the preset level corresponding to each pseudo-labeled image slice based on the first labeling information, the first identification result, and the second identification result; a sixth determining module, used to determine a second type of matching degree between the first, second, and third labeling information; a first establishing module, used to establish a preset selection degree model corresponding to the first level based on the first confidence level, the second confidence level, and the third confidence level; a second establishing module, used to establish a preset selection degree model corresponding to the second level based on the first confidence level, the second confidence level, the third confidence level, and the first type of matching degree; and a third establishing module, used to establish a preset selection degree model corresponding to the third level based on the first confidence level, the second confidence level, the third confidence level, the first type of matching degree, and the second type of matching degree.

[0117] Optionally, the fifth determining module includes: a first determining submodule, used to determine the total number of first characters in the first marker information, the total number of second characters in the second marker information, and the total number of third characters in the third marker information, respectively; a first matching submodule, used to match the first marker information and the second marker information to obtain the number of first characters; a second matching submodule, used to match the first marker information and the third marker information to obtain the number of second characters; a third matching submodule, used to match the second marker information and the third marker information to obtain the number of third characters; and a second determining submodule, used to determine the first matching based on the number of first characters and the total number of first characters. The system has several submodules: a first submodule for determining a first matching degree, a second matching degree for determining a second matching degree based on the number of first characters and the total number of second characters; a third submodule for determining a third matching degree based on the number of second characters and the total number of first characters, and a fourth matching degree based on the number of second characters and the total number of third characters; a fourth submodule for determining a fifth matching degree based on the number of third characters and the total number of second characters, and a sixth matching degree based on the number of third characters and the total number of third characters; and a first classification submodule for classifying the first, second, third, fourth, fifth, and sixth matching degrees into a first category of matching degrees.

[0118] Optionally, the sixth determining module includes: a fourth matching submodule, used to match the first marker information, the second marker information, and the third marker information to obtain the fourth character count; a fifth determining submodule, used to determine the seventh matching degree based on the fourth character count and the total first character count; a sixth determining submodule, used to determine the eighth matching degree based on the fourth character count and the total second character count; a seventh determining submodule, used to determine the ninth matching degree based on the fourth character count and the total third character count; and a second classification submodule, used to classify the seventh matching degree, the eighth matching degree, and the ninth matching degree into a second type of matching degree.

[0119] Optionally, the filtering unit includes: a seventh determining module, used to determine the first selection degree of each pseudo-marked image slice in the first level using a preset selection degree model corresponding to the first level; a first placing module, used to place pseudo-marked images with a first selection degree less than a first preset threshold into the image slice queue to be reviewed; an eighth determining module, used to determine the second selection degree of each pseudo-marked image slice in the second level using a preset selection degree model corresponding to the second level; a second placing module, used to place pseudo-marked images with a second selection degree less than a second preset threshold into the image slice queue to be reviewed; a ninth determining module, used to determine the third selection degree of each pseudo-marked image slice in the third level using a preset selection degree model corresponding to the third level; and a third placing module, used to place pseudo-marked images with a third selection degree less than a third preset threshold into the image slice queue to be reviewed.

[0120] The aforementioned verification device may also include a processor and a memory. The receiving unit 40, the first determining unit 41, the identifying unit 42, the second determining unit 43, the filtering unit 44, etc., are all stored in the memory as program units, and the processor executes the aforementioned program units stored in the memory to realize the corresponding functions.

[0121] The processor described above contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and kernel parameters can be adjusted to perform verification on each image slice in the image slice queue to be verified.

[0122] The aforementioned memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0123] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program having the following method steps: receiving an image slice review request, determining a first confidence level for each pseudo-labeled image slice in a set of pseudo-labeled image slices, identifying each pseudo-labeled image slice in the set of pseudo-labeled image slices using a first recognition model to obtain a first recognition result, and identifying each pseudo-labeled image slice in the set of pseudo-labeled image slices using a second recognition model to obtain a second recognition result, determining a preset level corresponding to each pseudo-labeled image slice based on the first labeling information, the first recognition result, and the second recognition result, filtering pseudo-labeled image slices in the corresponding preset level using a preset selection degree model to obtain a queue of image slices to be reviewed, and reviewing each image slice to be reviewed in the queue of image slices to be reviewed.

[0124] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the above-described image slicing verification method.

[0125] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the above-described image slicing verification method.

[0126] Figure 5 This is a hardware structure block diagram of an electronic device (or mobile device) for a verification method of image slicing according to an embodiment of the present invention. Figure 5As shown, the electronic device may include one or more processors 502 (shown as 502a, 502b, ..., 502n in the figure) 502 (processor 502 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 504 for storing data. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a keyboard, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 5 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, the electronic device may also include components that are more... Figure 5 The more or fewer components shown, or having the same Figure 5 The different configurations shown.

[0127] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0128] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0129] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0130] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0131] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0132] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0133] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for verifying image slices, characterized in that, include: Receive an image slice verification request, wherein the image slice verification request includes at least: a set of pseudo-marked image slices; Determine a first confidence level for each pseudo-labeled image slice in the set of pseudo-labeled image slices, wherein each pseudo-labeled image slice corresponds to first labeling information; A first recognition model is used to identify each of the pseudo-labeled image slices in the set of pseudo-labeled image slices to obtain a first recognition result, and a second recognition model is used to identify each of the pseudo-labeled image slices in the set of pseudo-labeled image slices to obtain a second recognition result, wherein the first recognition model and the second recognition model are both pre-trained recognition models; Based on the first marking information, the first recognition result, and the second recognition result, a preset level corresponding to each pseudo-marked image slice is determined, wherein each preset level corresponds to a preset selection degree model, and each preset level has a different priority, with the highest preset level having the highest priority; The pseudo-marked image slices in the corresponding preset level are filtered using the preset selection degree model to obtain a queue of image slices to be reviewed, and each image slice in the queue of image slices to be reviewed is reviewed. The first identification result includes at least: second labeling information, the second identification result includes at least: third labeling information, and the step of determining the preset level corresponding to each pseudo-labeled image slice based on the first labeling information, the first identification result, and the second identification result includes: The first marker information, the second marker information, and the third marker information are compared to obtain a comparison result; If the comparison result indicates that the first marker information, the second marker information, and the third marker information are all consistent, the preset level of the pseudo-marked image slice is determined to be the first level. If the comparison result indicates that any two of the first, second, and third marker information are consistent and inconsistent with the other marker information, the preset level of the pseudo-marked image slice is determined to be the second level. If the comparison result indicates that the first marker information, the second marker information, and the third marker information are all inconsistent, the preset level of the pseudo-marked image slice is determined to be the third level. The first identification result further includes: a second confidence level; the second identification result further includes: a third confidence level; and after determining the preset level corresponding to each pseudo-labeled image slice based on the first labeling information, the first identification result, and the second identification result, it further includes: Determine the first type of matching degree between any two of the first, second, and third marker information; Determine the second type of matching degree among the first tag information, the second tag information, and the third tag information; Based on the first confidence level, the second confidence level, and the third confidence level, a preset selection degree model corresponding to the first level is established; Based on the first confidence level, the second confidence level, the third confidence level, and the first type of matching degree, a preset selection degree model corresponding to the second level is established; Based on the first confidence level, the second confidence level, the third confidence level, the first type of matching degree, and the second type of matching degree, a preset selection degree model corresponding to the third level is established.

2. The verification method according to claim 1, characterized in that, The step of determining a first confidence level for each pseudo-labeled image slice in the set of pseudo-labeled image slices includes: The set of pseudo-labeled image slices is sampled to obtain sampled samples; Based on the sampled data, determine the character accuracy of the pseudo-labeled image slice set; The character accuracy is characterized as the first confidence level for each of the pseudo-labeled image slices.

3. The verification method according to claim 1, characterized in that, The step of determining the first type of matching degree between any two of the first tag information, the second tag information, and the third tag information includes: The total number of first characters in the first marker information, the total number of second characters in the second marker information, and the total number of third characters in the third marker information are determined respectively. Match the first marker information and the second marker information to obtain the first character count; The first marker information and the third marker information are matched to obtain the second character count; Match the second marker information and the third marker information to obtain the number of third characters; A first matching degree is determined based on the number of the first characters and the total number of the first characters, and a second matching degree is determined based on the number of the first characters and the total number of the second characters; A third matching degree is determined based on the number of the second character and the total number of the first character, and a fourth matching degree is determined based on the number of the second character and the total number of the third character. Based on the number of the third character and the total number of the second character, a fifth matching degree is determined, and based on the number of the third character and the total number of the third character, a sixth matching degree is determined; The first matching degree, the second matching degree, the third matching degree, the fourth matching degree, the fifth matching degree, and the sixth matching degree are classified as the first type of matching degree.

4. The verification method according to claim 3, characterized in that, The step of determining the second type of matching degree among the first tag information, the second tag information, and the third tag information includes: The fourth character count is obtained by matching the first marker information, the second marker information, and the third marker information. The seventh matching degree is determined based on the number of the fourth character and the total number of the first character; The eighth matching degree is determined based on the number of the fourth character and the total number of the second character; The ninth matching degree is determined based on the number of the fourth character and the total number of the third character; The seventh matching degree, the eighth matching degree, and the ninth matching degree are classified as the second type of matching degree.

5. The verification method according to claim 1, characterized in that, The step of using the preset selection degree model to filter the pseudo-labeled image slices corresponding to the preset level to obtain a queue of image slices to be reviewed includes: Using the preset selection model corresponding to the first level, the first selection degree of each pseudo-labeled image slice in the first level is determined; The pseudo-marked image with a first selection degree indicator that is less than the first preset threshold is sliced ​​and placed into the image slice queue to be reviewed; Using the preset selection model corresponding to the second level, the second selection degree of each pseudo-labeled image slice in the second level is determined; The pseudo-marked images whose second selection degree is less than the second preset threshold are sliced ​​and placed into the image slice queue to be reviewed. The third selectivity of each pseudo-labeled image slice in the third level is determined by using the preset selectivity model corresponding to the third level. The pseudo-marked image that is less than the third preset threshold, indicated by the third selectivity, is sliced ​​and placed into the image slice queue to be reviewed.

6. An image slice verification device, characterized in that, include: A receiving unit is configured to receive an image slice verification request, wherein the image slice verification request includes at least: a set of pseudo-marked image slices; The first determining unit is used to determine the first confidence level of each pseudo-labeled image slice in the set of pseudo-labeled image slices, wherein each pseudo-labeled image slice corresponds to first labeling information; The recognition unit is configured to use a first recognition model to recognize each of the pseudo-labeled image slices in the set of pseudo-labeled image slices to obtain a first recognition result, and use a second recognition model to recognize each of the pseudo-labeled image slices in the set of pseudo-labeled image slices to obtain a second recognition result, wherein the first recognition model and the second recognition model are both pre-trained recognition models; The second determining unit is used to determine a preset level corresponding to each pseudo-marked image slice based on the first marking information, the first recognition result and the second recognition result, wherein each preset level corresponds to a preset selection degree model, and each preset level has a different priority, with the highest preset level having the highest priority; The filtering unit is used to filter the pseudo-marked image slices in the corresponding preset level using the preset selection degree model to obtain a queue of image slices to be reviewed, and to review each image slice to be reviewed in the queue of image slices to be reviewed. The first identification result includes at least: second labeling information; the second identification result includes at least: third labeling information; the second determining unit includes: a first comparison module, used to compare the first labeling information, the second labeling information, and the third labeling information to obtain a comparison result; a second determining module, used to determine the preset level of the pseudo-labeled image slice as the first level when the comparison result indicates that the first labeling information, the second labeling information, and the third labeling information are all consistent; a third determining module, used to determine the preset level of the pseudo-labeled image slice as the second level when the comparison result indicates that any two of the first labeling information, the second labeling information, and the third labeling information are consistent and the other labeling information is inconsistent; and a fourth determining module, used to determine the preset level of the pseudo-labeled image slice as the third level when the comparison result indicates that the first labeling information, the second labeling information, and the third labeling information are all inconsistent. The first identification result further includes a second confidence level, and the second identification result further includes a third confidence level. The device further includes a fifth determining module, used to determine a first type of matching degree between any two of the first, second, and third labeling information after determining the preset level corresponding to each pseudo-labeled image slice based on the first labeling information, the first identification result, and the second identification result; a sixth determining module, used to determine a second type of matching degree between the first, second, and third labeling information; a first establishing module, used to establish a preset selection degree model corresponding to the first level based on the first confidence level, the second confidence level, and the third confidence level; a second establishing module, used to establish a preset selection degree model corresponding to the second level based on the first confidence level, the second confidence level, the third confidence level, and the first type of matching degree; and a third establishing module, used to establish a preset selection degree model corresponding to the third level based on the first confidence level, the second confidence level, the third confidence level, the first type of matching degree, and the second type of matching degree.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the image slice verification method according to any one of claims 1 to 5.

8. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the image slicing verification method according to any one of claims 1 to 5.

Images