Data-based cleaning method

Abstract

The invention provides a data-based cleaning method, and relates to a data processing technology, and the method comprises the steps: carrying out the classified scanning processing of pre-stored images in a database according to a preset scanning strategy, adding a corresponding scanning label to each type of pre-stored image, and enabling the scanning labels to at least comprise a character scanning label and a non-character scanning label; based on the retention probability information and / or the retention key fields, carrying out secondary classification extraction on the pre-stored images with the character scanning labels, and respectively adding corresponding probability labels and / or key word labels to the pre-stored images meeting the retention probability information and / or the retention key fields; performing secondary classification extraction on pre-stored images of non-text scanning labels based on the retained image targets, and adding corresponding image target labels to the pre-stored images meeting the retained image targets; pre-stored images without labels after secondary classification are cleaned and deleted, and the pre-stored images with different labels are classified and stored.

Description

Technical Field

[0001] The present invention relates to data processing technology, and in particular to a data-based cleaning method. Background Art

[0002] With the rapid development of big data and artificial intelligence technologies, image data is increasingly being used in various fields. However, due to the diverse sources and uneven quality of image data, how to efficiently clean image data and improve data quality and availability has become an urgent problem to be solved. Especially in the fields of finance, medicine, law, etc., image data may contain a large amount of text information (such as invoices, contracts, medical records, etc.). How to quickly identify, classify and extract effective information from massive images has become the core challenge of data cleaning. In addition, there is a large amount of non-text information in image data (such as graphics, icons, photos, etc.). How to distinguish between text and non-text content and perform refined processing according to user needs is also a key problem that current technology needs to solve.

[0003] Existing image data cleaning technologies mainly rely on optical character recognition (OCR) and simple label classification methods. For example, existing technologies use OCR to identify text content in images and classify and store images according to preset rules. However, existing technologies lack user-defined configuration functions and cannot dynamically adjust cleaning strategies according to user needs, resulting in a mismatch between cleaning results and user needs, reducing cleaning efficiency.

[0004] Therefore, how to dynamically adjust the cleaning strategy based on user needs and improve cleaning efficiency has become an urgent problem that needs to be solved. Summary of the invention

[0005] The embodiment of the present invention provides a data-based cleaning method, which can dynamically adjust the cleaning strategy in combination with user needs, meet the user's cleaning needs, and improve cleaning efficiency.

[0006] A first aspect of an embodiment of the present invention provides a data-based cleaning method, comprising: Classify and scan the pre-stored images in the database according to the preset scanning strategy, and add corresponding scanning labels to each category of pre-stored images, wherein the scanning labels at least include text scanning labels and non-text scanning labels; Generate a retention interaction interface corresponding to the current scan, and receive at least one of the retention text probability information, the retention key field, and the retention image target configured by the user based on the retention interaction interface; Based on the retention probability information and / or retention key fields, secondary classification and extraction are performed on the pre-stored images with text scanning labels, and corresponding probability labels and / or keyword labels are added to the pre-stored images that meet the retention probability information and / or retention key fields; Based on the retained image target, the pre-stored images with non-text scanned labels are secondary classified and extracted, and corresponding image target labels are added to the pre-stored images that meet the retained image target; The pre-stored images without labels after secondary classification are cleaned and deleted, and the pre-stored images with different labels are classified and stored.

[0007] Optionally, in a possible implementation manner of the first aspect, the pre-stored images in the database are classified and scanned according to a preset scanning strategy, and corresponding scanning labels are added to the pre-stored images of each category, wherein the scanning labels at least include text scanning labels and non-text scanning labels, including: Determine a scanning time period in a preset scanning strategy, determine a deadline based on the current time and the scanning time period, and generate a corresponding scanning extraction time based on the current time and the deadline; Extracting and processing the pre-stored images in the database based on the scanning extraction time to obtain a set of images to be scanned; Based on OCR, the pre-stored image recognition processing in the scanned image set is performed, and if the text is determined to be not empty, the corresponding text scanning label is added; if the text is determined to be empty, the corresponding non-text scanning label is added.

[0008] Optionally, in a possible implementation manner of the first aspect, generating a retention interaction interface corresponding to the current scan, and receiving at least one of retention text probability information, retention key fields, and retention image targets configured by a user based on the retention interaction interface, includes: Generate a retention interaction interface corresponding to the current scan, wherein the retention interaction interface includes at least a text probability slot, a key field slot, and a retention image slot; If it is determined that the corresponding text probability information is input into the text probability slot, a corresponding preview image is generated based on the text probability information, and the text probability information is saved after the user confirms the input; If it is determined that a key word phrase is input into the key field slot, the key word phrase is segmented and saved; If it is determined that a retained image target is input into the retained image slot, the retained image target is saved and processed; Calculation is performed based on the information in the text scanning labels and non-text scanning labels, text probability slots, key field slots, and retained image slots to obtain the estimated scanning information of the retained interactive interface.

[0009] Optionally, in a possible implementation manner of the first aspect, if it is determined that corresponding text probability information is input into the text probability slot, a corresponding preview image is generated based on the text probability information, and the text probability information is saved after determining that the user confirms the input, including: Generate an initial preview layer of a first layer specification, wherein the first layer specification includes pixel color, horizontal pixel number, and vertical pixel number; Obtaining a first text layer of text specifications corresponding to a text, wherein the first text specifications include pixel color, horizontal pixel number, and vertical pixel number; Calculate the range ratio of a first text layer relative to the preview layer, and divide the range ratio based on the text probability information to obtain the number of characters; Based on the number of characters, preset text information is called to obtain preview text information, and the preview text information is typeset and placed on the upper part of the preview layer to generate a preview image.

[0010] Optionally, in a possible implementation manner of the first aspect, calling preset text information based on the number of characters to obtain preview text information, and typeset and place the preview text information on an upper portion of a preview layer to generate a preview image includes: Determine a corresponding preview sub-template based on the number of characters, different intervals of the number of characters have preset preview sub-templates, and the preview sub-template includes the number of horizontal characters; The number of lines is obtained based on the number of horizontal characters in the preview sub-template and the number of characters calculated this time, and corresponding character slots are generated in the preview sub-template based on the number of lines, where each character corresponds to a character slot; The preview text information is sequentially filled into the text slots to obtain a layout image, and the center point of the layout image and the center point of the preview layer are overlapped and set to be placed on the upper part of the preview layer to generate a preview image.

[0011] Optionally, in a possible implementation manner of the first aspect, the calculating based on the information in the text scanning label and the non-text scanning label, the text probability slot, the key field slot, and the retained image slot to obtain the estimated scanning information of the retained interactive interface includes: Obtaining the number of first images having text scanning labels and the number of first images without text scanning labels in the set of images to be scanned; Extracting the text probability information in the text probability slot, obtaining the first field quantity of the field in the key field slot, and obtaining the second image quantity of the image corresponding to the retained image target in the retained image slot; The number of the first images, the word probability information, the number of the first fields, and the number of the second images are input as first parameters into a pre-trained calculation formula; The estimated scanning information of the historical retained interactive interface is extracted, and the second parameter is generated according to the attribute calculation processing of the historical retained interactive interface, and the second parameter is input into the pre-trained calculation formula to obtain the estimated scanning information of the retained interactive interface.

[0012] Optionally, in a possible implementation of the first aspect, extracting the estimated scanning information of the historical retained interactive interface, generating a second parameter according to the attribute calculation processing of the historical retained interactive interface, and inputting the second parameter into a pre-trained calculation formula to obtain the estimated scanning information of the retained interactive interface includes: Extract the estimated scanning information of the historical retained interactive interface, and obtain the first sub-time of calculating the probability of the text corresponding to each image, the second sub-time of scanning the corresponding field, and the third sub-time of comparing the historical retained image target when calculating the historical estimated scanning information for each retained interactive interface; The mean probability information and the mean number of fields of all historical retained interactive interfaces are calculated to obtain the mean probability and the mean number of fields, and the mean time corresponding to all first sub-times, second sub-times, and third sub-times is calculated to obtain the first sub-mean time, the second sub-mean time, and the third sub-mean time; The mean probability, the first sub-mean time, the field quantity mean, the second sub-mean time, and the third sub-mean time are input into the pre-trained calculation formula as the second parameter.

[0013] Optionally, in a possible implementation of the first aspect, extracting the estimated scanning information of the historical retained interactive interface, generating a second parameter according to the attribute calculation processing of the historical retained interactive interface, and inputting the second parameter into a pre-trained calculation formula to obtain the estimated scanning information of the retained interactive interface includes: The mean probability, first sub-mean time, field quantity mean, second sub-mean time, and third sub-mean time are calculated using the following formulas: in, is the mean probability, The first step in calculating the probability of words in history The text probability information of the retention interaction interface, The upper limit of the retention interface for calculating the historical text probability, The number of retained interactive interfaces for historical calculation of text probability, is the first sub-mean time, is the first sub-time of the history of calculating the probability of each image text, The upper limit of the retained interactive interface for calculating the probability of each image text in history, The number of retained interactive interfaces for calculating the probability of each image text for history, is the mean value of the number of fields, The number of historical calculation fields with the mean value The number of fields in the retention interface, The upper limit of the retention interface for the number of historical calculation fields. The number of values ​​in the retained interactive interface for the historical calculation field quantity. is the second sub-mean time, The first sub-time of the corresponding field of the historical scan, The upper limit of the retention interface for the historical scanned corresponding fields. The number of retained interactive interfaces for the corresponding fields of the historical scans. The upper limit of the retention interface for the historical scanned corresponding fields. is the third sub-mean time, For historical comparison, the image target is retained. The first sub-time of the retained interaction interface, The upper limit of the retention interface of the historical comparison historical retention image target, The number of retained interactive interfaces of the historically retained image targets for historical comparison; The estimated time value of scanning information is calculated by the following formula: in, Estimated time for the first child, The first image number of the scanned label for the text, Estimated time for the second child, This is the text probability information of the retained interactive interface. is the first constant value, is the first weight value, is the average number of fields in the retained interaction interface. is the second constant value, is the second weight value, Estimated time for the third child, is the first image number of images with non-text scanned labels, is the third weight value, It is the estimated time value of scanning information.

[0014] Optionally, in a possible implementation of the first aspect, the method further includes: Obtain the actual time when the user completes the current scan based on the retained interactive interface, and calculate the difference between the actual time and the time value of the estimated scan information to obtain the difference time; Obtaining a ratio of the number of first images of text scanning labels scanned this time to the number of first images of images with non-text scanning labels, to obtain a label ratio; The first weight value, the second weight value and the third weight value are subjected to differentiated training processing based on the difference time and the label ratio.

[0015] Optionally, in a possible implementation manner of the first aspect, performing differentiated training processing on the first weight value, the second weight value, and the third weight value based on the difference time and the label ratio includes: If the difference time is greater than 0, it is determined to call the forward training model, determine the differential adjustment range of the first weight value, the second weight value and the third weight value based on the label ratio, and perform down-adjustment training; If the difference time is less than 0, determine to call the reverse training model, determine the differential adjustment range of the first weight value, the second weight value and the third weight value based on the label ratio, and perform augmentation training; The first weight value, the second weight value, and the third weight value after reducing or increasing the training are calculated by the following formula: in, To increase the first weight value after training, The first image number of the scanned label for the text, is the first image number of images with non-text scanned labels, is the label weight value, is the weight value of non-text label, is the first offset coefficient value, To increase the second weight value after training, To increase the third weight value after training, is the second offset coefficient value, To reduce the first weight value after training, To reduce the second weight value after training, To reduce the third weight value after training.

[0016] The present invention realizes the automatic classification and labeling of image data through preset scanning strategies and OCR technology. Specifically, the system extracts a set of images to be scanned according to a preset scanning time period, and identifies the text content in the image through OCR technology. For images with recognized text, the system will add a "text scanning label" to them; for images without recognized text, a "non-text scanning label" will be added. This technical effect solves the problem of insufficient classification accuracy in the prior art and provides a reliable data basis for subsequent refined cleaning. In addition, through the preset scanning strategy, the system can dynamically adjust the scanning range according to user needs, further improving the flexibility and accuracy of classification.

[0017] The present invention is based on the dynamic configuration and preview function of the cleaning strategy of the user interaction interface, and realizes the dynamic configuration and preview function of the cleaning strategy by the user by generating a retention interaction interface. Specifically, the user can set the probability of retaining text, retaining key fields, and retaining image targets through the interface. The system generates a preview image according to the user configuration and saves the configuration information after the user confirms. This technical effect solves the problem of poor user interactivity in the prior art, and significantly improves the accuracy of the cleaning results and user satisfaction through the intuitive preview function and dynamic configuration mechanism. For example, the user can set through the interface to only retain images with a text probability greater than 80%, or only retain images containing the word "invoice", thereby realizing personalized customization of the cleaning strategy.

[0018] The present invention optimizes the estimated scanning information based on historical data and a dynamic weight adjustment mechanism, and optimizes the accuracy of the estimated scanning information by combining historical data and a dynamic weight adjustment mechanism. Specifically, the system extracts the estimated scanning information of the retained interactive interface from the historical data, generates a second parameter by mean calculation, and inputs it into a pre-trained calculation formula in combination with the first parameter (such as text probability information, number of fields, etc.) to obtain the current estimated scanning information. In addition, the system dynamically adjusts the first weight value, the second weight value, and the third weight value by calculating the difference between the actual scanning time and the estimated time and analyzing the label ratio, thereby further improving the accuracy of the estimated scanning information. This technical effect solves the problem of inefficiency in the prior art, and significantly improves the cleaning efficiency and real-time performance through the optimization of historical data and dynamic weight adjustment. For example, when there is a large difference between the actual scanning time and the estimated time, the system will automatically adjust the weight value to optimize the subsequent estimated scanning information. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flow chart of a data-based cleaning method provided by an embodiment of the present invention; Figure 2 It is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] See also Figure 1 , is a flow chart of a data-based cleaning method provided by an embodiment of the present invention, the method comprising: S1, classifying and scanning the pre-stored images in the database according to a preset scanning strategy, and adding corresponding scanning tags to each category of the pre-stored images, wherein the scanning tags at least include text scanning tags and non-text scanning tags.

[0022] With the rapid development of big data and artificial intelligence technology, image data is increasingly used in various fields. However, due to the diverse sources and uneven quality of image data, how to efficiently clean image data and improve data quality and availability has become an urgent problem to be solved.

[0023] In some embodiments, the pre-stored images in the database are classified and scanned according to a preset scanning strategy, and corresponding scanning tags are added to each category of pre-stored images, wherein the scanning tags at least include text scanning tags and non-text scanning tags, including: S11, determining a scanning time period in a preset scanning strategy, determining a deadline based on a current time and the scanning time period, and generating a corresponding scanning extraction time based on the current time and the deadline.

[0024] The preset scanning policy defines the time range of scanning (i.e. scanning period), which can be set according to user needs, such as the past month. The setting of scanning period and generation of extraction time can be realized through timestamp calculation or scheduled task scheduling tools.

[0025] S12, extracting the pre-stored images in the database based on the scanning extraction time to obtain a set of images to be scanned.

[0026] The system extracts pre-stored images that meet the conditions from the database based on the generated scan extraction time. For example, image data within the scan extraction time range (the past month) is extracted. The extracted image data is stored in a collection of images to be scanned for subsequent processing. S13, based on OCR, the pre-stored image recognition processing in the scanned image set is performed, and if it is determined that the text is not empty, a corresponding text scanning label is added; if it is determined that the text is empty, a corresponding non-text scanning label is added.

[0027] The system uses OCR (Optical Character Recognition) technology to perform text recognition on each image in the scanned image set. If OCR recognizes that the image contains text content (that is, the text is not empty), a "text scan label" is added to the image; if OCR does not recognize text content (that is, the text is empty), a "non-text scan label" is added to the image.

[0028] For example, assuming that a large number of images containing and not containing text are stored in the database, the system first extracts the set of images to be scanned according to the preset scanning time period, and then uses OCR technology to identify the text content in the image. For images with recognized text, the system will add a "text scan label" to them; for images without recognized text, a "non-text scan label" will be added. Subsequently, the system generates a retention interaction interface, through which users can set the probability of retaining text (for example, only retain images with a text probability greater than 80%) or specify key fields (for example, only retain images containing the word "invoice"), thereby achieving refined cleaning of image data.

[0029] Through preset scanning strategies and OCR technology, automatic classification and labeling of image data are achieved, which significantly improves the efficiency of data cleaning. At the same time, through the retention interactive interface, users can personalize the data cleaning process, which further improves the accuracy and practicality of data cleaning.

[0030] S2, generating a retention interaction interface corresponding to the current scan, and receiving at least one of the retention text probability information, the retention key field, and the retention image target configured by the user based on the retention interaction interface.

[0031] In step S2, a retention interaction interface corresponding to the current scan is generated, and at least one of the retention text probability information, retention key fields, and retention image targets configured by the user is received through the interface.

[0032] In some embodiments, the generating of the retention interaction interface corresponding to the current scan, receiving at least one of the retention text probability information, the retention key field, and the retention image target configured by the user based on the retention interaction interface, includes: S21, generating a retention interaction interface corresponding to the current scan, wherein the retention interaction interface at least includes a text probability slot, a key field slot, and a retention image slot.

[0033] The retention interaction interface includes text probability slots, key field slots, and retention image slots, through which users can enter relevant information. The system generates a graphical user interface (GUI) for interaction with users. The interface includes the following slots: text probability slot: used to enter retention text probability information (for example, only retain images with a text probability greater than 20%); key field slot: used to enter retention key fields (for example, only retain images containing the word "invoice"); retention image slot: used to specify retention image targets (for example, only retain images of person A). The user configuration is received through the interactive interface to achieve personalized customization of data cleaning.

[0034] S22: If it is determined that the corresponding text probability information is input into the text probability slot, a corresponding preview image is generated based on the text probability information, and the text probability information is saved after the user confirms the input.

[0035] If the user enters text probability information into the text probability slot, the system will generate a preview image based on the information and save the text probability information after the user confirms it. This step enables the user to personalize the data cleaning process through the interactive interface, improving the flexibility and accuracy of data cleaning.

[0036] If the user enters text probability information (e.g. 20%) in the text probability slot, the system will generate an image that meets the conditions based on the information and generate a preview image for the user to view. After the user confirms that the preview image meets expectations, the system will save the text probability information and apply it to the subsequent data cleaning process.

[0037] If it is determined that the corresponding text probability information is input into the text probability slot, a corresponding preview image is generated based on the text probability information, and the text probability information is saved after the user confirms the determination, including: S221, generating an initial preview layer of a first layer specification, wherein the first layer specification includes pixel color, horizontal pixel number, and vertical pixel number.

[0038] The system generates an initial preview layer with specific specifications, where the pixel color is used to define the color value of each pixel in the layer (e.g. RGB format). The horizontal number of pixels defines the width of the layer. The vertical number of pixels defines the height of the layer. You can create a blank image object through an image processing library (such as OpenCV or PIL) and set its size and background color. Generate a standardized preview layer as the basis for subsequent text layer overlays.

[0039] S222, obtaining a first text layer of text specifications corresponding to a text, wherein the first text specifications include pixel color, horizontal pixel quantity, and vertical pixel quantity.

[0040] The system obtains a text layer, which contains the rendering results of a single text, where the pixel color defines the color value of the text (e.g. black RGB (0,0,0)), which needs to be distinguished from the pixel value of the preview layer, for example, the pixel value of the preview layer can be white. The horizontal number of pixels defines the width of the text (e.g. 50 pixels). The vertical number of pixels defines the height of the text (e.g. 50 pixels). Obtain the rendering results of a single text to provide basic data for subsequent calculation of the number of texts.

[0041] S223, calculating a range ratio of a first text layer relative to the preview layer, and performing division calculation based on the text probability information and the range ratio to obtain the number of characters.

[0042] The system calculates the range ratio of a single text layer relative to the preview layer. The specific steps are as follows: The area of ​​a single text layer is the text width × text height, and the area of ​​the preview layer is the layer width × layer height. The range ratio calculation formula is: range ratio = (text width × text height) / (layer width × layer height). Based on the text probability information entered by the user (for example, 20%), combined with the range ratio, the number of texts to be displayed in the preview layer is calculated. The number of texts in the preview layer is dynamically adjusted based on the range ratio and text probability information to ensure that the preview image meets the user configuration.

[0043] S224, based on the number of characters, calling preset text information to obtain preview text information, and typeset the preview text information and place it on the upper part of the preview layer to generate a preview image.

[0044] Based on the calculated number of words, the system calls the preset text information (such as "sample text"), generates preview text information, and typesets it and overlays it on the top of the preview layer. It extracts text content (such as "sample text") from the preset text library and repeatedly generates enough text information based on the number of words. The generated text information is arranged on the top of the preview layer according to certain typesetting rules (such as from left to right, from top to bottom). The typeset text information is overlaid on the preview layer to generate the final preview image. By generating a preview image, the effect corresponding to the text probability information configured by the user is intuitively displayed, which is convenient for the user to confirm and adjust.

[0045] The method of calling preset text information based on the number of characters to obtain preview text information, and typeset the preview text information and place it on the upper part of the preview layer to generate a preview image includes: S2241, determining a corresponding preview sub-template based on the number of characters, different intervals of the number of characters having preset preview sub-templates, the preview sub-template including the number of horizontal characters.

[0046] The system selects a suitable sub-template from the preset preview sub-template library based on the calculated number of characters. The system predefines multiple preview sub-templates, each of which corresponds to a character number range. For example: Character number range 1 (1-10 characters): the horizontal number of characters is 5. Character number range 2 (11-20 characters): the horizontal number of characters is 10. Character number range 3 (21-30 characters): the horizontal number of characters is 15. According to the calculated number of characters, the corresponding character number range is matched and the corresponding preview sub-template is selected.

[0047] S2242. Obtain the number of lines based on the number of horizontal characters in the preview sub-template and the number of characters calculated this time. Generate corresponding text slots within the preview sub-template based on the number of lines, with each character corresponding to one text slot.

[0048] The system calculates the required number of lines based on the number of horizontal characters in the preview sub-template and the number of characters calculated this time, and generates corresponding text slots. The formula for calculating the number of lines is: number of lines = ceil(number of characters / number of horizontal characters). Here, ceil means rounding up. Generate corresponding text slots within the preview sub-template according to the number of lines and the number of horizontal characters. By generating text slots, it provides a structured layout for subsequent text filling and typesetting.

[0049] S2243. Fill the preview text information into the text slots in sequence to obtain a typeset image. After setting the center point of the typeset image to overlap with the center point of the preview layer, place it above the preview layer to generate a preview image.

[0050] The system fills the preview text information into the text slots in sequence to generate a typeset image, and superimposes it above the preview layer. Fill the preset preview text information (such as "example text") into each text slot in sequence. Combine the filled text slots into a complete typeset image. Determine the center point of the typeset image and the center point of the preview layer, and align them to overlap. Superimpose the typeset image above the preview layer to generate the final preview image. Through the center point alignment and superimposition operations, ensure that the typesetting of the preview image is beautiful and meets the user's expectations.

[0051] S23. If it is determined that a keyword sentence is input to the keyword field slot, perform word segmentation on the keyword sentence and then save it.

[0052] When the system detects that the user inputs a keyword sentence to the keyword field slot, the system performs word segmentation on the keyword sentence and saves it. The system monitors the keyword field slot in real time to determine whether a keyword sentence is input. Perform word segmentation on the input keyword sentence, specifically including: using a word segmentation tool (such as Jieba, HanLP, etc.) to split the keyword sentence into independent words or phrases. Remove stop words (such as meaningless words like "of", "is", etc.). Retain the words or phrases with practical meanings. Save the result after word segmentation to the storage module of the system for subsequent calculation and analysis. Word segmentation processing can be implemented through natural language processing (NLP) technology, and the word segmentation results can be saved using a database or cache. Through word segmentation processing, extract the core information in the keyword sentence to provide data support for subsequent estimated scan information calculation.

[0053] S24. If it is determined that a retained image target is input to the retained image slot, perform save processing on the retained image target.

[0054] When the system detects that the user inputs a retained image target into the retained image slot, the system saves the retained image target. The system monitors the retained image slot in real time to determine whether a retained image target is input. The retained image target is saved to the storage module of the system. The image is compressed or converted to a format (such as PNG or JPEG) to optimize storage space. By saving the retained image target, image data support is provided for subsequent estimated scan information calculation.

[0055] S25, performing calculations based on the text scanning labels and non-text scanning labels, the text probability slots, the key field slots, and the information in the retained image slots to obtain estimated scanning information of the retained interactive interface.

[0056] The system performs comprehensive calculations based on the information in text scanning tags, non-text scanning tags, text probability slots, key field slots, and retained image slots to obtain the estimated scanning information of the retained interactive interface. Through comprehensive calculations, the system provides users with the estimated scanning information of the retained interactive interface to help users optimize interface design and content layout.

[0057] The calculation based on the information in the text scanning label and the non-text scanning label, the text probability slot, the key field slot and the retained image slot to obtain the estimated scanning information of the retained interactive interface includes: S251, obtaining the number of first images having text scanning labels and the number of first images without text scanning labels in the set of images to be scanned.

[0058] The system obtains the number of images with text scanning labels and non-text scanning labels from the set of images to be scanned. The set of images to be scanned is obtained, which contains multiple image targets. According to whether the image target contains text content, it is divided into two categories: Text scanning label: Indicates that the image contains text content. Non-text scanning label: Indicates that the image does not contain text content (such as graphics, icons, etc.). Count the number of images with text scanning labels and the number of images with non-text scanning labels. By counting the number of text and non-text images, basic data is provided for the subsequent calculation of estimated scanning information.

[0059] S252, extracting the text probability information in the text probability slot, obtaining the first field quantity of the field in the key field slot, and obtaining the second image quantity of the image corresponding to the retained image target in the retained image slot.

[0060] The system extracts relevant information from the text probability slot, key field slot, and retained image slot. Extract text probability information from the text probability slot, which indicates the probability value of the image containing text content (such as a value between 0.0-1.0). Obtain the number of fields from the key field slot (recorded as the first field number), which indicates the number of key fields saved after word segmentation. Obtain the number of images corresponding to the retained image target from the retained image slot (recorded as the second image number). The above information can be extracted by database query or cache reading. By extracting text probability information, the number of fields, and the number of images, multi-dimensional data support is provided for subsequent estimated scanning information calculations.

[0061] S253: Input the first number of images, the text probability information, the first number of fields, and the second number of images as first parameters into a pre-trained calculation formula.

[0062] The system uses the number of first images, text probability information, the number of first fields, and the number of second images obtained in steps S251 and S252 as the first parameter, and inputs the first parameter into a pre-trained calculation formula, which is used to calculate the estimated scan information of the retained interactive interface. By inputting the first parameter, basic data is provided for the calculation of the estimated scan information. The formula for text calculation is provided below.

[0063] S254, extracting the estimated scanning information of the historical retained interactive interface, generating a second parameter based on the attribute calculation processing of the historical retained interactive interface, and inputting the second parameter into a pre-trained calculation formula to obtain the estimated scanning information of the retained interactive interface.

[0064] Extract the estimated scan information of the retained interactive interface from the historical data of the system. Calculate and generate the second parameter based on the attributes of the historical data. Input the second parameter into the pre-trained calculation formula and calculate the estimated scan information of the current retained interactive interface together with the first parameter. By introducing historical data, the accuracy and reliability of the estimated scan information are improved.

[0065] The extracting of the estimated scanning information of the historical retained interactive interface, generating a second parameter according to the attribute calculation processing of the historical retained interactive interface, and inputting the second parameter into the pre-trained calculation formula to obtain the estimated scanning information of the retained interactive interface includes: S2541, extract the estimated scanning information of the historical retained interactive interface, and obtain the first sub-time of calculating the probability of text corresponding to each image, the second sub-time of scanning the corresponding field, and the third sub-time of comparing the historical retained image target when calculating the historical estimated scanning information of each retained interactive interface.

[0066] The system extracts the estimated scanning information of the retained interactive interface from the historical data and obtains the time parameters involved in the calculation process. The estimated scanning information of the retained interactive interface is extracted from the historical data of the system, including text probability information, number of fields, and number of retained images. For each retained interactive interface, the following time parameters are obtained: First sub-time: the time to calculate the probability of the text corresponding to each image. Second sub-time: the time to scan the corresponding field. Third sub-time: the time to compare the historical retained image targets. By obtaining the time parameters, basic data is provided for the subsequent mean calculation.

[0067] S2542, calculate the mean probability and the mean number of fields of all historical retained interactive interfaces, and calculate the mean time corresponding to all first sub-times, second sub-times, and third sub-times to obtain the first sub-mean time, second sub-mean time, and third sub-mean time.

[0068] The system calculates the mean of the text probability information, field number, and time parameters in the historical data. The text probability information of all historical retained interactive interfaces is averaged to obtain the mean probability. The field number of all historical retained interactive interfaces is averaged to obtain the mean field number. The first sub-time, second sub-time, and third sub-time of all historical data are averaged to obtain: First sub-mean time: mean time for calculating text probability. Second sub-mean time: mean time for scanning fields. Third sub-mean time: mean time for comparing retained image targets. Through mean calculation, noise in historical data is eliminated and more representative parameters are extracted.

[0069] S2543, input the mean probability, the first sub-mean time, the field quantity mean, the second sub-mean time, and the third sub-mean time as second parameters into the pre-trained calculation formula.

[0070] The system uses the mean probability, first sub-mean time, field number mean, second sub-mean time, and third sub-mean time calculated in step S2542 as the second parameter and inputs them into the pre-trained calculation formula. The following parameters are integrated into the second parameter: mean probability (mean of historical text probability information, such as the mean of the past three query histories). First sub-mean time (mean time for calculating text probability). Field number mean (mean of historical field number). Second sub-mean time (mean time for scanning fields). Third sub-mean time (mean time for comparing retained image targets). By inputting the second parameter and combining the mean information of historical data, the accuracy and reliability of the estimated scanning information can be improved.

[0071] In some embodiments, the extracting of the estimated scanning information of the historical retained interactive interface, generating a second parameter based on the attribute calculation processing of the historical retained interactive interface, and inputting the second parameter into a pre-trained calculation formula to obtain the estimated scanning information of the retained interactive interface includes: The mean probability, first sub-mean time, field quantity mean, second sub-mean time, and third sub-mean time are calculated using the following formulas: in, is the mean probability, The first step in calculating the probability of words in history The text probability information of the retention interaction interface, The upper limit of the retention interface for calculating the historical text probability, The number of retained interactive interfaces for historical calculation of text probability, is the first sub-mean time, is the first sub-time of the history of calculating the probability of each image text, The upper limit of the retained interactive interface for calculating the probability of each image text in history, The number of retained interactive interfaces for calculating the probability of each image text for history, is the mean value of the number of fields, The number of historical calculation fields with the mean value The number of fields in the retention interface, The upper limit of the retention interface for the number of historical calculation fields. The number of values ​​in the retained interactive interface for the historical calculation field quantity. is the second sub-mean time, The first sub-time of the corresponding field of the historical scan, The upper limit of the retention interface for the historical scanned corresponding fields. The number of retained interactive interfaces for the corresponding fields of the historical scans. The upper limit of the retention interface for the historical scanned corresponding fields. is the third sub-mean time, For historical comparison, the image target is retained. The first sub-time of the retained interaction interface, The upper limit of the retention interface of the historical comparison historical retention image target, The number of retained interactive interfaces for historical comparisons of image targets In the above formula, To calculate the mean value of the text probability information of the historically retained interactive interface; The average time to calculate the probability of text in the historical retention interaction interface; To calculate the average number of fields in the historically retained interactive interface as a reference value for estimating scan information; To calculate the average time of scanning fields in the historical retention interaction interface as the basis for the estimated time, where: is the first sub-time of the historical scan of the corresponding field, is the average time of the scan in each historical time period, and then the second sub-average time is calculated for each historical time period. To calculate the average time of comparing the retained image targets in the historical retained interaction interface as the basis for the estimated time.

[0072] The estimated time value of scanning information is calculated by the following formula: in, Estimated time for the first child, The first image number of the scanned label for the text, Estimated time for the second child, This is the text probability information of the retained interactive interface. is the first constant value, is the first weight value, is the average number of fields in the retained interaction interface. is the second constant value, is the second weight value, Estimated time for the third child, is the first image number of images with non-text scanned labels, is the third weight value, It is the estimated time value of scanning information.

[0073] In the above formula, To calculate the estimated text probability calculation time based on the difference between the text probability information of the current retention interaction interface and the historical average probability, where: The larger the weight is, the greater the difference is, so the first sub-estimated time needs to be larger. The first weight value can be preset by the user. To calculate the estimated field scanning time based on the difference between the average number of fields in the current retained interaction interface and the average number of historical fields, The larger the weight is, the larger the difference is, so the larger the second sub-estimated time needs to be. The second weight value can be preset by the user. The estimated comparison time is calculated based on the average time of historical comparison of the retained image targets, wherein the third weight value can be preset by the user. Need to be greater than -1 and less than 1, and It needs to be greater than -1 and less than 1. For example, and Set it larger.

[0074] On the basis of the above embodiment, an optimization mechanism for the estimated scanning information is further introduced, and the first weight value, the second weight value and the third weight value are dynamically adjusted by calculating the difference between the actual scanning time and the estimated time and analyzing the label ratio, so as to improve the accuracy of the estimated scanning information. It also includes: The actual time when the user completes the current scan based on the retained interactive interface is obtained, and the difference between the actual time and the time value of the estimated scan information is calculated to obtain the difference time.

[0075] Get the actual completion time from the user's saved interactive interface for this scan. The actual time can be obtained through system logs, timestamps, or other time recording tools. Calculate the difference between the actual time and the estimated time to evaluate the accuracy of the estimate. If the difference time is greater than 0, it means that the estimated time is too short and the actual time is too long. If the difference time is less than 0, it means that the estimated time is too long and the actual time is too short.

[0076] The ratio of the number of first images with text scanning labels scanned this time to the number of first images with non-text scanning labels is obtained to obtain a label ratio.

[0077] The following data is obtained from this scan: the number of first images with text scan labels, the number of first images with non-text scan labels, and the label ratio, which represents the ratio of the number of images with text scan labels to the number of images with non-text scan labels. Through label ratio analysis, the distribution of text and non-text content in this scan is evaluated to provide a basis for weight adjustment.

[0078] The first weight value, the second weight value and the third weight value are subjected to differentiated training processing based on the difference time and the label ratio.

[0079] The differential training process of the first weight value, the second weight value and the third weight value based on the difference time and the label ratio includes: If the difference time is greater than 0, it is determined to call the forward training model, determine the differential adjustment range of the first weight value, the second weight value and the third weight value based on the label ratio, and perform down-training. It can be understood that if the difference time is greater than 0, it means that the actual time is greater than the estimated time, which means that the estimated time is too long and the weight value needs to be reduced to reduce the estimated time. Through the forward training model, the weight value is reduced for training.

[0080] If the difference time is less than 0, it is determined to call the reverse training model, determine the differential adjustment range of the first weight value, the second weight value and the third weight value based on the label ratio, and perform increase training. It can be understood that if the difference time is less than 0, it means that the actual time is less than the estimated time, which means that the estimated time is too long and the weight value needs to be increased to improve the estimated time. Through the reverse training model, the weight value is increased.

[0081] The first weight value, the second weight value, and the third weight value after reducing or increasing the training are calculated by the following formula: in, To increase the first weight value after training, The first image number of the scanned label for the text, is the first image number of images with non-text scanned labels, is the label weight value, is the weight value of non-text label, is the first offset coefficient value, To increase the second weight value after training, To increase the third weight value after training, is the second offset coefficient value, To reduce the first weight value after training, To reduce the second weight value after training, To reduce the third weight value after training.

[0082] In the above formula, If it is greater than 1, it means that the text accounts for a large proportion. In this case, the first weight value and the second weight value corresponding to the text need to be adjusted to a larger value. Therefore, ; If it is less than 1, the description text accounts for a small proportion. In this case, the third weight value corresponding to the image needs to be adjusted more. Therefore, The text label weight value and the non-text label weight value may be preset by the user. The first offset coefficient value and the second offset coefficient value may be preset by the user.

[0083] S3, based on the retention probability information and / or retention key fields, secondary classification and extraction of pre-stored images with text scanning labels, and adding corresponding probability labels and / or keyword labels to the pre-stored images that meet the retention probability information and / or retention key fields.

[0084] Perform secondary classification on the pre-stored images with text scan labels, and extract images that meet the following conditions: Retention probability information: The image's retention probability value is higher than the preset threshold. Retention key fields: The image contains preset key fields (such as specific text, symbols, or information). Add the following labels to the images that meet the conditions: Probability label: A label generated based on retention probability information; Keyword label: A label generated based on retention key fields (for example, contains "contract", contains "invoice", etc.). Through secondary classification extraction and label addition, the classification of images with text scan labels is further refined to facilitate subsequent classification storage and retrieval.

[0085] S4, based on the retained image target, secondary classification and extraction of pre-stored images with non-text scan labels, and adding corresponding image target labels to the pre-stored images that meet the retained image target.

[0086] Perform secondary classification on pre-stored images with non-text scanned labels and extract images that meet the following conditions: Retained image target: The image contains the preset target object or feature (for example, face, vehicle, building, etc.). Add the following labels to the images that meet the conditions: Image target label: The label generated based on the retained image target (for example, containing "face A", etc.). Through secondary classification extraction and label addition, the classification of images with non-text scanned labels is further refined to facilitate subsequent classification storage and retrieval.

[0087] S5, cleaning and deleting the pre-stored images without labels after secondary classification, and classifying and storing the pre-stored images with different labels.

[0088] Clean and delete the pre-stored images that do not have any labels after secondary classification. Delete invalid or irrelevant image data to reduce storage space usage and improve data quality. Classify and store pre-stored images with different labels. Implement orderly management of image data to facilitate subsequent retrieval, analysis and application. Through secondary classification extraction and label addition, fine classification of pre-stored images is achieved to improve the efficiency of data management. By cleaning and deleting images without labels, the storage of invalid data is reduced and the use of storage resources is optimized. Through classified storage, orderly management of image data is achieved to facilitate subsequent retrieval and application. Through the above steps, the present invention can achieve fine classification, label addition and classified storage of pre-stored images, and at the same time has data cleaning functions, which is suitable for a variety of image management and analysis scenarios.

[0089] See also Figure 2 , is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention, the electronic device 20 includes: a processor 21, a memory 22 and a computer program; wherein The memory 22 is used to store the computer program, and the memory may also be a flash memory. The computer program is, for example, an application program, a functional module, etc. for implementing the above method.

[0090] The processor 21 is used to execute the computer program stored in the memory to implement each step performed by the device in the above method. For details, please refer to the relevant description in the above method embodiment.

[0091] Optionally, the memory 22 may be independent or integrated with the processor 21 .

[0092] When the memory 22 is a device independent of the processor 21, the device may further include: The bus 23 is used to connect the memory 22 and the processor 21 .

[0093] The present invention also provides a storage medium, in which a computer program is stored. When the computer program is executed by a processor, it is used to implement the methods provided by the various embodiments described above.

[0094] Among them, the storage medium can be a computer storage medium or a communication medium. The communication medium includes any medium that facilitates the transmission of a computer program from one place to another. The computer storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer. For example, the storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application-specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). In addition, the ASIC can be located in a user device. Of course, the processor and the storage medium can also exist in a communication device as discrete components. The storage medium can be a read-only memory (ROM), a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0095] The present invention also provides a program product, which includes an execution instruction, which is stored in a storage medium. At least one processor of a device can read the execution instruction from the storage medium, and at least one processor executes the execution instruction so that the device implements the methods provided in the above various embodiments.

[0096] In the above-mentioned terminal or server embodiments, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the present invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0097] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data-based cleaning method, characterized in that: include: Classify and scan the pre-stored images in the database according to the preset scanning strategy, and add corresponding scanning labels to each category of pre-stored images, wherein the scanning labels at least include text scanning labels and non-text scanning labels; Generate a retention interaction interface corresponding to the current scan, and receive at least one of the retention text probability information, the retention key field, and the retention image target configured by the user based on the retention interaction interface; Based on the retention probability information and / or retention key fields, secondary classification and extraction are performed on the pre-stored images with text scanning labels, and corresponding probability labels and / or keyword labels are added to the pre-stored images that meet the retention probability information and / or retention key fields; Based on the retained image target, the pre-stored images with non-text scanned labels are secondary classified and extracted, and corresponding image target labels are added to the pre-stored images that meet the retained image target; The pre-stored images without labels after secondary classification are cleaned and deleted, and the pre-stored images with different labels are classified and stored.

2. The data-based cleaning method according to claim 1, characterized in that: The pre-stored images in the database are classified and scanned according to the preset scanning strategy, and corresponding scanning labels are added to the pre-stored images of each category, wherein the scanning labels at least include text scanning labels and non-text scanning labels, including: Determine a scanning time period in a preset scanning strategy, determine a deadline based on the current time and the scanning time period, and generate a corresponding scanning extraction time based on the current time and the deadline; Extracting and processing the pre-stored images in the database based on the scanning extraction time to obtain a set of images to be scanned; Based on OCR, the pre-stored image recognition processing in the scanned image set is performed, and if the text is determined to be not empty, the corresponding text scanning label is added; if the text is determined to be empty, the corresponding non-text scanning label is added.

3. The data-based cleaning method according to claim 1, characterized in that: The generating of a retention interaction interface corresponding to the current scan, and receiving at least one of retention text probability information, retention key fields, and retention image targets configured by a user based on the retention interaction interface, includes: Generate a retention interaction interface corresponding to the current scan, wherein the retention interaction interface includes at least a text probability slot, a key field slot, and a retention image slot; If it is determined that the corresponding text probability information is input into the text probability slot, a corresponding preview image is generated based on the text probability information, and the text probability information is saved after the user confirms the input; If it is determined that a key word phrase is input into the key field slot, the key word phrase is segmented and saved; If it is determined that a retained image target is input into the retained image slot, the retained image target is saved and processed; Calculation is performed based on the information in the text scanning labels and non-text scanning labels, text probability slots, key field slots, and retained image slots to obtain the estimated scanning information of the retained interactive interface.

4. The data-based cleaning method according to claim 3, characterized in that: If it is determined that the corresponding text probability information is input into the text probability slot, a corresponding preview image is generated based on the text probability information, and the text probability information is saved after determining that the user confirms, including: Generate an initial preview layer of a first layer specification, wherein the first layer specification includes pixel color, horizontal pixel number, and vertical pixel number; Obtaining a first text layer of text specifications corresponding to a text, wherein the first text specifications include pixel color, horizontal pixel number, and vertical pixel number; Calculate the range ratio of a first text layer relative to the preview layer, and divide the range ratio based on the text probability information to obtain the number of characters; Based on the number of characters, preset text information is called to obtain preview text information, and the preview text information is typeset and placed on the upper part of the preview layer to generate a preview image.

5. The data-based cleaning method according to claim 4, characterized in that: The method of calling preset text information based on the number of characters to obtain preview text information, and typeset the preview text information and place it on the upper part of the preview layer to generate a preview image includes: Determine a corresponding preview sub-template based on the number of characters, different intervals of the number of characters have preset preview sub-templates, and the preview sub-template includes the number of horizontal characters; The number of lines is obtained based on the number of horizontal characters in the preview sub-template and the number of characters calculated this time, and corresponding character slots are generated in the preview sub-template based on the number of lines, with each character corresponding to one character slot; The preview text information is sequentially filled into the text slots for typeset to obtain a typeset image, and the center point of the typeset image and the center point of the preview layer are overlapped and set to be placed on the upper part of the preview layer to generate a preview image.

6. The data-based cleaning method according to claim 4, characterized in that: The calculation is performed based on the information in the text scanning label and the non-text scanning label, the text probability slot, the key field slot and the retained image slot to obtain the estimated scanning information of the retained interactive interface, including: Obtaining the number of first images having text scanning labels and the number of first images without text scanning labels in the set of images to be scanned; Extracting the text probability information in the text probability slot, obtaining the first field quantity of the field in the key field slot, and obtaining the second image quantity of the image corresponding to the retained image target in the retained image slot; The first number of images, the word probability information, the first number of fields, and the second number of images are input as first parameters into a pre-trained calculation formula; The estimated scanning information of the historical retained interactive interface is extracted, and the second parameter is generated according to the attribute calculation processing of the historical retained interactive interface, and the second parameter is input into the pre-trained calculation formula to obtain the estimated scanning information of the retained interactive interface.

7. The data-based cleaning method according to claim 6, characterized in that: The extracting of the estimated scanning information of the historical retained interactive interface, generating a second parameter according to the attribute calculation processing of the historical retained interactive interface, and inputting the second parameter into the pre-trained calculation formula to obtain the estimated scanning information of the retained interactive interface, includes: Extract the estimated scanning information of the historical retained interactive interface, and obtain the first sub-time of calculating the probability of the text corresponding to each image, the second sub-time of scanning the corresponding field, and the third sub-time of comparing the historical retained image target when calculating the historical estimated scanning information for each retained interactive interface; The mean probability information and the mean number of fields of all historical retained interactive interfaces are calculated to obtain the mean probability and the mean number of fields, and the mean time corresponding to all first sub-times, second sub-times, and third sub-times is calculated to obtain the first sub-mean time, the second sub-mean time, and the third sub-mean time; The mean probability, the first sub-mean time, the field quantity mean, the second sub-mean time, and the third sub-mean time are input as second parameters into a pre-trained calculation formula.

8. The data-based cleaning method according to claim 7, characterized in that: The extracting of the estimated scanning information of the historical retained interactive interface, generating a second parameter according to the attribute calculation processing of the historical retained interactive interface, and inputting the second parameter into the pre-trained calculation formula to obtain the estimated scanning information of the retained interactive interface, includes: The mean probability, first sub-mean time, field quantity mean, second sub-mean time, and third sub-mean time are calculated using the following formulas: in, is the mean probability, The first step in calculating the probability of words in history The text probability information of the retention interaction interface, The upper limit of the retention interface for calculating the historical text probability, The number of retained interactive interfaces for historical calculation of text probability, is the first sub-mean time, is the first sub-time of the history of calculating the probability of each image text, The upper limit of the retained interactive interface for calculating the probability of each image text in history, The number of retained interactive interfaces for calculating the probability of each image text for history, is the mean value of the number of fields, The number of historical calculation fields with the mean value The number of fields in the retention interface, The upper limit of the retention interface for the number of historical calculation fields. The number of values ​​in the retained interactive interface for the historical calculation field quantity. is the second sub-mean time, The first sub-time of the corresponding field of the historical scan, The upper limit of the retention interface for the historical scanned corresponding fields. The number of retained interactive interfaces for the corresponding fields of the historical scans. The upper limit of the retention interface for the historical scanned corresponding fields. is the third sub-mean time, For historical comparison, the image target is retained. The first sub-time of the retained interaction interface, The upper limit of the retention interface of the historical comparison historical retention image target, The number of retained interactive interfaces of the historically retained image targets for historical comparison; The estimated time value of scanning information is calculated by the following formula: in, Estimated time for the first child, The first image number of the scanned label for the text, Estimated time for the second child, This is the text probability information of the retained interactive interface. is the first constant value, is the first weight value, is the average number of fields in the retained interaction interface. is the second constant value, is the second weight value, Estimated time for the third child, is the first image number of images with non-text scanned labels, is the third weight value, It is the estimated time value of scanning information.

9. The data-based cleaning method according to claim 8, characterized in that: Also includes: Obtain the actual time when the user completes the current scan based on the retained interactive interface, and calculate the difference between the actual time and the time value of the estimated scan information to obtain the difference time; Obtaining a ratio of the number of first images of text scanning labels scanned this time to the number of first images of images with non-text scanning labels, to obtain a label ratio; The first weight value, the second weight value and the third weight value are subjected to differential training processing based on the difference time and the label ratio.

10. The data-based cleaning method according to claim 9, characterized in that: The performing differentiated training processing on the first weight value, the second weight value, and the third weight value based on the difference time and the label ratio includes: If the difference time is greater than 0, it is determined to call the forward training model, determine the differential adjustment range of the first weight value, the second weight value and the third weight value based on the label ratio, and perform down-adjustment training; If the difference time is less than 0, determine to call the reverse training model, determine the differential adjustment range of the first weight value, the second weight value and the third weight value based on the label ratio, and perform augmentation training; The first weight value, the second weight value, and the third weight value after reducing or increasing the training are calculated by the following formula: in, To increase the first weight value after training, The first image number of the scanned label for the text, is the first image number of images with non-text scanned labels, is the label weight value, is the weight value of non-text label, is the first offset coefficient value, To increase the second weight value after training, To increase the third weight value after training, is the second offset coefficient value, To reduce the first weight value after training, To reduce the second weight value after training, To reduce the third weight value after training.

Images