Data collection, text recognition method, device, equipment and storage medium

Abstract

The application provides a data collection method, a text recognition method, a device, equipment and a storage medium. The data collection method comprises the following steps: obtaining community publishing data of a user; performing a first comparison between the community publishing data and original user data in a user data set of the user; determining original user data that is successfully compared in the first comparison as first recognition data; performing a second comparison between remaining community publishing data, i.e., community publishing data that fails in the first comparison, and a preset total data set; determining original historical data that is successfully compared in the second comparison as second recognition data; counting a current occurrence frequency of the first recognition data and a current occurrence frequency of the second recognition data; adding second recognition data with a current occurrence frequency greater than an occurrence frequency threshold to the user data set to collect user data of the user. The data collection method is highly targeted for a single user. When a text recognition is performed on the user data set obtained by using the data collection method, the text recognition is highly accurate and highly targeted.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, specifically to a data collection and text recognition method, apparatus, device, and storage medium. Background Technology

[0002] In recent years, with the widespread adoption of smartphones and other devices, and the rise of numerous social media platforms, more and more people are choosing to share their thoughts online. As the intelligent connected vehicle business develops, OEMs will also provide diverse open platforms. These platforms can offer one-stop services including design, development, verification, testing, and experience sharing, allowing developers to fully enjoy the advantages of atomization and open standards while also meeting the needs for stability and security. For the long-term healthy operation of the open platform, it is necessary to review published articles and comments to ensure that their content does not violate laws, regulations, or social ethics.

[0003] However, the data involved on open platforms is quite complex, including but not limited to images and text. Text recognition is required during the review process. Therefore, the accuracy of text recognition becomes particularly important. However, existing text recognition technologies often have poor applicability and low accuracy for individual users. Therefore, there is an urgent need to improve the targeting and accuracy of individual user data collection methods. Summary of the Invention

[0004] In view of the shortcomings of the prior art described above, embodiments of the present invention provide a data collection method, apparatus, device, and storage medium to solve the above-mentioned technical problems.

[0005] The data collection method provided in this invention includes: acquiring community-posted data of users; performing a first comparison between the community-posted data and the user's user dataset, identifying the original user data in the user dataset that successfully matches the first comparison as first identification data, the user dataset including multiple original user data and the frequency of occurrence of the original user data; performing a second comparison between the community-posted data that fails the first comparison and a preset total dataset, identifying the original historical data in the preset total dataset that successfully matches the second comparison as second identification data, the preset total dataset including multiple original historical data and the frequency of occurrence of the original historical data; calculating the current occurrence frequency of the first identification data and the current occurrence frequency of the second identification data based on the occurrence frequency of the original user data and the occurrence frequency of the original historical data; adding the second identification data whose current occurrence frequency is greater than an occurrence frequency threshold to the user dataset to collect the user's user data.

[0006] In one embodiment of the present invention, after performing a second comparison between the community-published data that failed the first comparison and a preset total dataset, and before calculating the current occurrence frequency of the first identified data and the current occurrence frequency of the second identified data based on the occurrence frequency of the original user data and the occurrence frequency of the original historical data, the data collection method further includes: performing text recognition on the community-published data that failed the second comparison; adding the recognition result of the text recognition to the preset total dataset; and determining the recognition result as the second identified data.

[0007] In one embodiment of the present invention, the original user data includes historical posting data and historical user identification results of the historical posting data. A first comparison is performed between the community posting data and the user's user dataset. Determining the original user data in the successfully matched user dataset as the first identification data includes: performing a first comparison between the community posting data and the historical posting data; if the first similarity between the community posting data and the historical posting data is greater than a first preset similarity threshold, the first comparison result is determined as a successful first comparison; and the historical user identification results of the historical posting data where the first similarity is greater than the first preset similarity threshold are determined as the first identification data.

[0008] In one embodiment of the present invention, the original historical data includes the original published data and the original identification result of the original published data. The community published data that failed the first comparison is compared with a preset total dataset in the second comparison. The original historical data in the preset total dataset that succeeded in the second comparison is determined as the second identification data. This includes: comparing the community published data with the original published data in the second comparison; if the second similarity between the community published data and the original published data is greater than a second preset similarity threshold, the second comparison result is determined as a second comparison success; and the original identification result of the original published data whose second similarity is greater than the second preset similarity threshold is determined as the second identification data.

[0009] In one embodiment of the present invention, before performing a second comparison between the community-published data that failed the first comparison and the preset total dataset, the data collection method further includes: acquiring user datasets of multiple users; generating the preset total dataset based on the multiple user datasets, wherein the original historical data is determined based on the original user data of each user dataset.

[0010] In one embodiment of the present invention, adding the second identification data whose current occurrence frequency is greater than the occurrence frequency threshold to the user dataset includes: associating the original identification result, the community-published data and the current occurrence frequency of the second identification data, and adding it to the user dataset.

[0011] In one embodiment of the present invention, adding the second identification data whose current occurrence frequency is greater than the occurrence frequency threshold to the user dataset includes: obtaining the data storage information of the original historical data corresponding to the second identification data; and adding the data storage information to the user dataset.

[0012] In one embodiment of the present invention, before adding the recognition result of text recognition to the preset total dataset, the method further includes: obtaining a preset disabled text set; comparing the preset disabled text set with the recognition result of text recognition; and if the recognition result passes the comparison, adding the passing recognition result to the preset total dataset.

[0013] In one embodiment of the present invention, after adding the second identification data whose current occurrence frequency is greater than the occurrence frequency threshold to the user dataset, the method further includes at least one of the following: determining the user's user preference data based on the occurrence frequency of the original user data in the user dataset, the user preference data including at least one original user data; determining the community's community preference data based on the occurrence frequency of the original historical data in the preset total dataset, the community preference information including at least one of the original historical data.

[0014] In one embodiment of the present invention, the second identification data whose current occurrence frequency is greater than an occurrence frequency threshold is added to the user dataset. The data collection method further includes:

[0015] Sort the current occurrence frequency of the first identification data and the current occurrence frequency of the second identification data;

[0016] The current occurrence frequency of the first identified data or the current occurrence frequency of the second identified data, whose sorting value is a preset order value, is determined as the occurrence frequency threshold. An embodiment of the present invention provides a text recognition method, the text recognition method comprising: acquiring user's community data to be identified; performing a third comparison between the community data to be identified and the user's user dataset, determining the original user data in the user dataset that successfully completes the third comparison as third identified data, the original user data in the user dataset being collected by the data collection method described in any of the above embodiments; performing a fourth comparison between the community data to be identified that fails the third comparison and a preset total dataset, determining the original historical data in the preset total dataset that successfully completes the fourth comparison as fourth identified data, the preset total dataset including user datasets of multiple users; and generating a text recognition result for the community data to be identified based on the third identified data and the fourth identified data.

[0017] This invention provides a data collection device, comprising: an acquisition module for acquiring community-posted data of users; a first comparison module for performing a first comparison between the community-posted data and the user's user dataset, identifying existing user data in the user dataset that successfully completes the first comparison as first identification data, the user dataset including multiple existing user data and the frequency of occurrence of existing user data; a second comparison module for performing a second comparison between the community-posted data that fails the first comparison and a preset total dataset, identifying existing historical data in the preset total dataset that successfully completes the second comparison as second identification data, the preset total dataset including multiple existing historical data and the frequency of occurrence of existing historical data; a statistics module for calculating the current occurrence frequency of the first identification data and the current occurrence frequency of the second identification data based on the occurrence frequency of the existing user data and the occurrence frequency of the existing historical data; and a determination module for adding second identification data whose current occurrence frequency is greater than an occurrence frequency threshold to the user dataset, thereby collecting the user's user data.

[0018] An electronic device provided in this invention includes: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the electronic device enables the method described in any of the above embodiments.

[0019] The present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a computer processor, causes the computer to perform the method described in any of the above embodiments.

[0020] The beneficial effects of the embodiments of the present invention are as follows: The data collection, text recognition method, apparatus, device, and storage medium in the embodiments of the present invention collect user community posting data, compare the community posting data with the original user data in the user's own user dataset, and determine the original user data that successfully matches in the first comparison as the first recognition data. Then, the remaining community posting data, that is, the community posting data that failed in the first comparison, is compared with a preset total dataset in the second comparison, and the original historical data that successfully matches in the second comparison is determined as the second recognition data. The current occurrence frequency of the first recognition data and the current occurrence frequency of the second recognition data are counted. The second recognition data whose current occurrence frequency is greater than the occurrence frequency threshold is added to the user dataset to collect user data. It is highly targeted to a single user. When the user dataset obtained by this data collection method is used for text recognition, the text recognition accuracy is high and the targeting is good.

[0021] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0023] Figure 1 This is a flowchart illustrating a data collection method in an exemplary embodiment of this application;

[0024] Figure 2 This is a flowchart illustrating a specific data collection method in an exemplary embodiment of this application;

[0025] Figure 3 This is a flowchart illustrating a text recognition method in an exemplary embodiment of this application;

[0026] Figure 4 This is a block diagram illustrating a data collection apparatus according to an exemplary embodiment of this application;

[0027] Figure 5 This is a block diagram illustrating a text recognition device in an exemplary embodiment of this application;

[0028] Figure 6 This is a block diagram illustrating a text recognition device in an exemplary embodiment of this application;

[0029] Figure 7 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0030] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0031] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0032] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0033] In recent years, with the widespread adoption of smartphones and other devices, and the rise of numerous social media platforms, more and more people are choosing to share their thoughts online. With the development of the intelligent connected vehicle business, OEMs are offering a wide range of capabilities and collaborating more frequently with the external ecosystem. Creating an open platform will enable OEMs to provide unified, complete, stable, and efficient external capabilities, and to manage ecosystem resources in a unified manner. OEMs will also provide a diverse open platform that offers one-stop services including design, development, verification, testing, and experience sharing, allowing developers to fully enjoy the advantages of atomization and open standards while also meeting the needs for stability and security. For the long-term healthy operation of the open platform, published articles and comments need to be reviewed to ensure that their content does not violate laws, regulations, or social ethics.

[0034] However, the data involved on open platforms is quite complex, including but not limited to images and text. Text recognition is required during the review process. Therefore, the accuracy of text recognition becomes particularly important. However, existing text recognition technologies often have poor applicability and low accuracy for individual users. Therefore, there is an urgent need to improve the targeting and accuracy of individual user data collection methods.

[0035] To address the aforementioned problems, embodiments of this application propose a data collection method, a data collection device, a text recognition method, an electronic device, a computer-readable storage medium, and a computer program product, which will be described in detail below.

[0036] Please see Figure 1 , Figure 1 This is a flowchart illustrating a data collection method as shown in an exemplary embodiment of this application. This method can be performed in an implementation environment known to those skilled in the art. Figure 1As shown, in an exemplary embodiment, the data collection method includes at least steps S101 to S105, which are described in detail below:

[0037] Step S101: Obtain the user's community posting data.

[0038] Users can be engineers, developers, or ordinary netizens who publish public content on the open platform. Community-published data refers to publicly available content posted on the open platform, including but not limited to at least one of the following: publicly published comments, posts, images, and videos. Text data can be extracted and / or identified from the community-published data.

[0039] This method can be implemented through a server or a client, or a combination of both.

[0040] Before releasing data, the community must obtain the consent of relevant personnel and departments, and obtain the data in a reasonable and legal manner and through legal channels.

[0041] The data released by the community can be one or more pieces of content.

[0042] Step S102: Perform a first comparison between the community-published data and the user dataset, and determine the original user data in the user dataset that is successfully matched as the first identification data.

[0043] The user dataset includes multiple sets of existing user data and their frequencies. Since people's comments on a particular type of event often reflect certain viewpoints, these viewpoints tend to be consistent, or people often have common phrases, catchphrases, or frequently used images. This content is constantly posted by users on open platforms. By collecting this content data and adding it to the user dataset as existing user data, and recording the frequency of its occurrence, it's easier to analyze the user's habits and preferences later. The frequency of image occurrences can be directly used as the existing user data frequency, or the text or image recognition results can be used as the existing user data frequency. Different images may produce the same text recognition results; in this case, the existing user data frequencies can be accumulated. Then, based on the same text recognition results, the community posting data that yielded those results, and the existing user data frequency, the existing user data is generated.

[0044] The user dataset is initially empty, so the first alignment will fail.

[0045] Since the data published in the community may contain multiple elements, such as multiple images or multiple paragraphs of text, there may be at least one original user data that is identical to a portion of the published data. In this case, the historical user identification result from that original user data can be determined as the first identification data. The determination principle for the second identification data is similar to that for the first identification data, and will not be elaborated upon further.

[0046] Since community-published data may take the form of text, images, etc., the corresponding type can be found in the existing user data for initial comparison based on the type of community-published data. This reduces the number of comparisons and lowers resource consumption.

[0047] In one embodiment, the original user data includes historical posting data and historical user identification results of the historical posting data. A first comparison is performed between the community posting data and the user dataset. The original user data in the successfully matched user dataset is identified as the first identification data, including:

[0048] The community-released data is first compared with historical release data;

[0049] If the first similarity between the community-published data and the historically published data is greater than the first preset similarity threshold, the first comparison result will be determined as the first comparison success.

[0050] The historical user identification results of historical published data with a first similarity greater than a first preset similarity threshold are determined as the first identification data.

[0051] Among them, the historical user identification results are the text recognition results of historical published data.

[0052] Using the above method, text recognition is not required every time. For content that the current user has previously recognized, the historical recognition content (historical user recognition results) can be used directly, saving text recognition time and reducing resource consumption.

[0053] Step S103: Perform a second comparison between the community-published data that failed the first comparison and the preset total dataset, and determine the original historical data in the preset total dataset that succeeded in the second comparison as the second identification data.

[0054] The preset total dataset can be a collection of user datasets containing multiple users. The preset total dataset is initially empty, at which point the second alignment will fail.

[0055] Step S104: Based on the frequency of occurrence of existing user data and the frequency of occurrence of existing historical data, calculate the current frequency of occurrence of the first identification data and the current frequency of occurrence of the second identification data.

[0056] Among them, the frequency of occurrence of original user data is the number of times that historical post data appears in the user's historical community post data. The frequency of occurrence of original historical data is the total number of times original post data appears in the historical community post data of all users in the community. For example, if user A's historical post data 1 appears 5 times and user B's historical post data 1 appears 9 times, in the total dataset, this historical post data is called original post data, and its original historical data frequency is 14 times.

[0057] For example, in the data released by the community, the original user data corresponding to the first identification data appeared 2 times. In the data released by the community, the first identification data appeared 3 times. Therefore, the current frequency of the first identification data after statistics is 5 times. The current frequency of the second identification data is similar, and will not be elaborated here.

[0058] For example, if a user posts two identical images in a community post, and the text recognition result is "happy", then the original user data containing the image appears 3 times in the user dataset. In this case, the current frequency of the first recognition data is 5 times.

[0059] In one embodiment, after performing a second comparison between the community-published data that failed the first comparison and a preset total dataset, and before calculating the current occurrence frequency of the first identification data and the current occurrence frequency of the second identification data based on the occurrence frequency of the original user data and the occurrence frequency of the original historical data, the data collection method further includes:

[0060] Text recognition was performed on the community-posted data that failed the second comparison.

[0061] The text recognition results are added to the preset total dataset, and the recognition results are designated as the second recognition data.

[0062] As mentioned above, the preset total dataset can be a collection of user datasets including multiple users. When adding text recognition results to the preset total dataset, these results can be stored in a preset temporary subset within the preset total dataset. This preset temporary subset does not belong to any user and is only used to store the recognition results. Once the data collection for the community's published data is complete, this preset temporary subset will be cleared. This saves storage space.

[0063] In one embodiment, before performing a second comparison between the community-published data that failed the first comparison and a preset total dataset, the data collection method further includes:

[0064] Obtain user datasets from multiple users;

[0065] A preset total dataset is generated based on multiple user datasets, and the original historical data is determined based on the original user data of each user dataset.

[0066] In other words, the original historical data and the original user data are consistent. For ease of description, the data is given different names in different dataset description contexts.

[0067] In one embodiment, the original historical data includes the original published data and the original identification results of the original published data. The community published data that failed the first comparison is compared with a preset total dataset in a second comparison. The original historical data in the preset total dataset that successfully completes the second comparison is determined as the second identification data, including:

[0068] A second comparison will be made between the data released by the community and the original data released.

[0069] If the second similarity between the community-published data and the original published data is greater than the second preset similarity threshold, the second comparison result will be determined as a successful second comparison.

[0070] The original identification results of the original published data with a second similarity greater than the second preset similarity threshold are determined as the second identification data.

[0071] Since the data published in the community may contain multiple elements, such as multiple images or multiple paragraphs of text, there may be at least one original data that is identical to a part of the data published in the community. In this case, the original identification result in the original data can be identified as the second identification data.

[0072] The original recognition result refers to the text recognition result of the original published data.

[0073] Using the above method, text recognition is not required every time. For content that has been recognized by other users before, the historical recognition content (the original recognition result) can be used directly, saving text recognition time and reducing resource consumption.

[0074] Step S105: Add the second identification data whose current occurrence frequency is greater than the occurrence frequency threshold to the user dataset to collect user data.

[0075] The frequency threshold is determined based on the current frequency of occurrence of the first identification data and the current frequency of occurrence of the second identification data.

[0076] In one embodiment, before adding second identification data whose current occurrence frequency is greater than an occurrence frequency threshold to the user dataset, the data collection method further includes:

[0077] Sort the current occurrence frequency of the first identification data and the current occurrence frequency of the second identification data;

[0078] The current occurrence frequency of the first identification data or the current occurrence frequency of the second identification data is determined as the occurrence frequency threshold.

[0079] For example, the current occurrence frequency of the first identification data and the current occurrence frequency of the second identification data are sorted from high to low. The current occurrence frequency of the first or second identification data ranked X is determined as the occurrence frequency threshold. That is, the top n% of the first identification data are added to the user dataset. X can be determined based on the current occurrence frequency of the first identification data and the number of data points of the second identification data. For example, if the number of data points is 10, X is 4.

[0080] By adding newly published community posts by a user that are not present in the user dataset, the data types in the user dataset can be enriched, which facilitates subsequent analysis of the user's language habits based on the user dataset and can improve the text recognition speed of the user's community posts.

[0081] In one embodiment, adding second identification data whose current occurrence frequency is greater than an occurrence frequency threshold to the user dataset includes:

[0082] Associate the current occurrence frequency of the original recognition results, community-published data, and second recognition data, and add them to the user dataset.

[0083] That is, the original identification results in other users' user datasets are associated with the current user-side community-published data and the current occurrence frequency of the updated second identification data, and stored in the user datasets for later use.

[0084] In one embodiment, adding second identification data whose current occurrence frequency is greater than an occurrence frequency threshold to the user dataset includes:

[0085] Obtain the data storage information of the original historical data corresponding to the second identification data;

[0086] Add data storage information to the user dataset.

[0087] Data storage information can include links to existing historical data, addresses, etc. This avoids repeatedly storing secondary identification data, saving overall data storage space.

[0088] In one embodiment, before adding the text recognition results to a preset total dataset, the method further includes:

[0089] Get the preset set of disabled texts;

[0090] The preset set of disabled texts is compared with the recognition results of text recognition.

[0091] If the recognition result passes the comparison with the system disabled, the passing recognition result will be added to the preset total dataset.

[0092] If the identification result fails the comparison, the identification result will not be added to the preset total dataset. A reminder can also be issued, or the number of times the identification result is identified can be accumulated. When a certain number of times is reached, the community administrators will be prompted to issue a relevant announcement to clarify the use of prohibited words.

[0093] The preset disabled text set can also be a subset of the preset total dataset.

[0094] In one embodiment, after adding second identification data whose current occurrence frequency is greater than an occurrence frequency threshold to the user dataset, the method further includes at least one of the following:

[0095] User preference data is determined based on the frequency of occurrence of existing user data in the user dataset. The user preference data includes at least one existing user data.

[0096] Community preference data is determined based on the frequency of occurrence of existing historical data in a preset total dataset. The community preference information includes at least one existing historical data.

[0097] For example, original user data whose frequency exceeds a certain threshold can be identified as user preference data, and original historical data whose frequency exceeds a certain threshold can be identified as community preference data.

[0098] Community preference data and user preference data make it easier for operators and community managers to analyze and understand the discussion intensity in the community.

[0099] User datasets can be stored in each user's user database, and preset total datasets can be stored in a preset total database. Users can pre-set user identification information to identify the source users of data published in the community.

[0100] The data collection method of this application is illustrated below through an exemplary embodiment. Please refer to [link to embodiment]. Figure 2 , Figure 2 This is a flowchart illustrating a specific data collection method as shown in an exemplary embodiment of this application. Figure 2As shown, a general text database (master database) is pre-created, and user information such as developers is assigned identification information, such as numbering each developer. A separate text database (user database) is then established and bound to each developer. Data within the community (community-published data, i.e., customer data in the diagram) is acquired and linked to the developers who published it. It is then determined which developer (user) published the data, and the user is tagged, resulting in community-published data tagged with user information. First, the community-published data is compared and identified with the developer's text database (i.e., the comparison analysis in the figure). The first identified text (first identification data, identification data 1) is saved, and the identified data is removed. Then, the remaining data (community-published data that failed the first comparison, unidentified data) is compared with the total text database to identify the text (second identification data, identification data 2). The community-published data (the part of community-published data corresponding to the second identification data) is bound to the identified text, and the number of texts is classified and counted. Texts with the same occurrences are grouped together. Texts with a high frequency of occurrence are extracted from the classified texts and recorded in the developer's separate text database.

[0101] The overall text database, also known as the total database, includes "image-text" and "text-text", with "text-text" containing multiple languages.

[0102] The overall text database also includes a prohibited database, which contains a preset set of prohibited texts, including but not limited to data that violates national laws and regulations, as well as vulgar language.

[0103] The above embodiments provide a data collection method. By acquiring community posting data from users, the method performs a first comparison with the original user data in the user's own user dataset. The original user data that successfully matches in the first comparison is identified as the first identification data. Then, the remaining community posting data, i.e., the community posting data that fails in the first comparison, is compared with a preset total dataset. The original historical data that successfully matches in the second comparison is identified as the second identification data. The current occurrence frequency of the first identification data and the current occurrence frequency of the second identification data are counted. The second identification data whose current occurrence frequency is greater than the occurrence frequency threshold is added to the user dataset to collect user data. This method is highly targeted to a single user. When the user dataset obtained by this data collection method is used for text recognition, the text recognition accuracy is high and the targeting is good.

[0104] Please see Figure 3 , Figure 3This is a flowchart illustrating a text recognition method in an exemplary embodiment of this application. This method can be performed in an implementation environment known to those skilled in the art. Figure 3 As shown, in an exemplary embodiment, the text recognition method includes at least steps S301 to S304, which are described in detail below:

[0105] Step S301: Obtain the user's community data to be identified.

[0106] The community data to be identified is similar to the community posting data in the above embodiments, and can be the community posting data newly published by the user, which will not be described in detail here.

[0107] Step S302: Perform a third comparison between the community data to be identified and the user dataset, and determine the original user data in the user dataset that has been successfully matched as the third identification data.

[0108] The original user data in the user dataset is collected by the data collection method described in any of the above embodiments, which will not be elaborated here. The related beneficial effects can also be referred to the above embodiments, which will not be elaborated here.

[0109] The implementation of this step can refer to the implementation of step S102 in the above embodiment, and will not be repeated here.

[0110] Step S303: Perform a fourth comparison between the community data to be identified that failed the third comparison and the preset total dataset, and determine the original historical data in the preset total dataset that succeeded in the fourth comparison as the fourth identification data.

[0111] The default total dataset includes user datasets from multiple users.

[0112] The implementation of this step can refer to the implementation of step S103 in the above embodiment, and will not be repeated here.

[0113] Step S304: Generate text recognition results for the community data to be identified based on the third and fourth identification data.

[0114] If there are still unidentified community data to be identified, they can be identified using methods known to those skilled in the art to ensure the completeness of the identification.

[0115] The above method allows for pre-processing of text recognition on user-posted community data to be detected within the user's own user dataset and the community's total dataset, resulting in better targeting, faster speed, and lower cost.

[0116] Figure 4 This is a block diagram illustrating a data collection apparatus as shown in an exemplary embodiment of this application. Figure 4As shown, the exemplary data collection device 400 includes:

[0117] The first acquisition module 401 is used to acquire user community posting data;

[0118] The first comparison module 402 is used to perform a first comparison between the community-published data and the user's user dataset, and to determine the original user data in the user dataset that is successfully matched as the first identification data. The user dataset includes multiple original user data and the frequency of occurrence of the original user data.

[0119] The second comparison module 403 is used to perform a second comparison between the community posting data that failed the first comparison and the preset total dataset, and to determine the original historical data in the preset total dataset that succeeded in the second comparison as the second identification data. The preset total dataset includes multiple original historical data and the frequency of occurrence of the original historical data.

[0120] The statistics module 404 is used to calculate the current occurrence frequency of the first identification data and the current occurrence frequency of the second identification data based on the occurrence frequency of the original user data and the occurrence frequency of the original historical data.

[0121] The determination module 405 is used to add second identification data whose current occurrence frequency is greater than the occurrence frequency threshold to the user dataset in order to collect user data.

[0122] It should be noted that the data collection device provided in the above embodiments is different from that in the above embodiments. Figure 2 The provided data collection methods belong to the same concept, and the specific ways in which each module and unit performs its operations have been described in detail in the method embodiments, and will not be repeated here. In practical applications, the data collection device provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above, and this is not a limitation.

[0123] Figure 5 This is a block diagram illustrating a text recognition device in an exemplary embodiment of this application. Figure 5 As shown, the exemplary text recognition device 500 includes:

[0124] The second acquisition module 501 is used to acquire the user's community data to be identified;

[0125] The third comparison module 502 is used to perform a third comparison between the community data to be identified and the user's user dataset, and to determine the original user data in the user dataset that has been successfully compared as the third identification data. The original user data in the user dataset is collected by the data collection method as described in any one of claims 1-9.

[0126] The fourth comparison module 503 is used to perform a fourth comparison between the community data to be identified that failed the third comparison and the preset total dataset. The original historical data in the preset total dataset that succeeded in the fourth comparison is determined as the fourth identification data. The preset total dataset includes user datasets of multiple users.

[0127] The generation module 504 is used to generate text recognition results of the community data to be identified based on the third and fourth recognition data.

[0128] It should be noted that the text recognition device provided in the above embodiments is different from that in the above embodiments. Figure 3 The provided text recognition methods belong to the same concept, and the specific ways in which each module and unit performs its operations have been described in detail in the method embodiments, and will not be repeated here. In practical applications, the text recognition device provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above, and this is not a limitation.

[0129] The following is an exemplary description of the application device of the text recognition method provided in this embodiment of the invention through another exemplary use scenario. See also Figure 6 , Figure 6 This is a block diagram illustrating a text recognition device in an exemplary embodiment of this application. Figure 6 As shown, the text recognition device includes:

[0130] The storage unit is used to store the existing overall "image-text" database as well as the user's personal database. That is, it is used to store user datasets and preset overall datasets, etc.

[0131] The data acquisition unit collects user data, such as data from the community to be identified.

[0132] The tagging unit tags the collected data with user information, that is, it distinguishes which user the community data to be identified belongs to.

[0133] The comparison unit compares the collected data with the personal database on the storage unit, and then with the total database. In other words, it performs a third comparison between the community data to be identified and the user's user dataset. The original user data in the user dataset that is successfully matched in the third comparison is determined as the third identification data.

[0134] The reprocessing unit is used to perform a fourth comparison between the community data to be identified that failed the third comparison and the preset total dataset, and to determine the original historical data in the preset total dataset that succeeded in the fourth comparison as the fourth identification data.

[0135] The text recognition results of the community data to be identified are generated using the third and fourth identification data.

[0136] The statistical unit performs statistics on the compared data and text. Specifically, it calculates the current occurrence frequency of the third and fourth identification data based on the occurrence frequency of the original user data and the occurrence frequency of the original historical data. The fourth identification data whose current occurrence frequency of the third identification data is greater than the occurrence frequency threshold is added to the user dataset to collect user data.

[0137] The reprocessing unit is also used to separate the identified data from the unidentified data.

[0138] The text recognition device described above collects frequently used texts based on users' personal habits and records the corresponding data. This allows for faster identification of high-frequency texts in user data during subsequent analysis, thus accelerating the data recognition process.

[0139] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. Multiple modules, units, and components may be combined or integrated into another apparatus or system, or some features may be ignored or not executed.

[0140] The communication connection described or implied in this application may be an indirect coupling or communication connection through some interface, device or unit, and may be electrical, mechanical or other forms.

[0141] Whether the modules or units described in the separate component description are physically separated is not limited here. Components with display functions can be physical units, that is, they can be located in one place or distributed across multiple network units. Some or all of the units or modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0142] Embodiments of this application also provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the methods provided in the above embodiments.

[0143] Figure 7 A schematic diagram of a computer system suitable for implementing the embodiments of this application is shown. It should be noted that... Figure 7The computer system 700 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0144] like Figure 7 As shown, the computer system 700 includes a Central Processing Unit (CPU) 701, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 702 or programs loaded from storage portion 708 into Random Access Memory (RAM) 703, such as performing the methods described in the above embodiments. The RAM 703 also stores various programs and data required for system operation. The CPU 701, ROM 702, and RAM 703 are interconnected via a bus 704. An Input / Output (I / O) interface 705 is also connected to the bus 704.

[0145] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0146] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1109, and / or installed from removable medium 711. When the computer program is executed by central processing unit (CPU) 701, it performs various functions defined in the system of this application.

[0147] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0148] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0149] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0150] Another aspect of this application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer's processor, causes the computer to perform the methods described in the preceding embodiments. This computer-readable storage medium may be included in the electronic devices described in the above embodiments, or it may exist independently and not assembled into the electronic device.

[0151] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various embodiments described above.

[0152] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

[0153] It should be noted that in this application, terms such as "first" and "second" are merely for distinguishing similar objects, and do not limit the order or sequence of similar objects. The variations of "including" and "having" indicate that the scope covered by the subject of the word is not exclusive, except for the examples shown by the word.

[0154] It is understood that the various numerical designations, step numbers, and other identifiers recorded in this application are for descriptive convenience and are not intended to limit the scope of this application. The size of the identifiers in this application does not imply the order of execution; the execution order of each process should be determined by its function and internal logic.

Claims

1. A data collection method, characterized in that, The data collection method includes: Obtain user community posting data; A first comparison is made between the community-published data and the user's user dataset. The original user data in the user dataset that successfully matches the first comparison is determined as the first identification data. The user dataset includes multiple original user data and the frequency of occurrence of the original user data. The community-published data that failed the first comparison is compared with a preset total dataset. The original historical data in the preset total dataset that succeeded in the second comparison is determined as the second identification data. The preset total dataset includes multiple original historical data and the frequency of occurrence of the original historical data. Based on the frequency of occurrence of the original user data and the frequency of occurrence of the original historical data, the current frequency of occurrence of the first identification data and the current frequency of occurrence of the second identification data are calculated. Sort the current occurrence frequency of the first identification data and the current occurrence frequency of the second identification data; The current occurrence frequency of the first identification data or the current occurrence frequency of the second identification data, whose sorting value is a preset order value, is determined as the occurrence frequency threshold. The second identification data whose current occurrence frequency is greater than the occurrence frequency threshold is added to the user dataset to collect the user's user data.

2. The data collection method as described in claim 1, characterized in that, After performing a second comparison between the community-published data that failed the first comparison and the preset total dataset, and before calculating the current occurrence frequency of the first identified data and the current occurrence frequency of the second identified data based on the occurrence frequency of the original user data and the occurrence frequency of the original historical data, the data collection method further includes: Text recognition is performed on the community-posted data that failed the second comparison; The text recognition result is added to the preset total dataset, and the recognition result is determined as the second recognition data.

3. The data collection method as described in claim 1, characterized in that, The existing user data includes historical posting data and historical user identification results of the historical posting data. A first comparison is performed between the community posting data and the user dataset. The existing user data in the user dataset that successfully matches the first comparison is identified as the first identification data, including: The community-posted data is compared with the historical posting data in the first comparison. If the first similarity between the community-published data and the historically published data is greater than a first preset similarity threshold, the first comparison result is determined as a successful first comparison. The historical user identification results of historical published data with a similarity greater than a first preset similarity threshold are determined as the first identification data.

4. The data collection method as described in claim 1, characterized in that, The original historical data includes the original published data and the original identification results of the original published data. The community published data that failed the first comparison is compared with the preset total dataset in the second comparison. The original historical data in the preset total dataset that succeeded in the second comparison is determined as the second identification data, including: A second comparison is made between the community-published data and the original published data; If the second similarity between the community-published data and the original published data is greater than the second preset similarity threshold, the second comparison result is determined as a successful second comparison. The original identification results of the original published data whose second similarity is greater than the second preset similarity threshold are determined as the second identification data.

5. The data collection method according to any one of claims 1-4, characterized in that, Before performing a second comparison between the community-published data that failed the first comparison and the preset total dataset, the data collection method further includes: Obtain user datasets from multiple users; The preset total dataset is generated based on multiple user datasets, and the original historical data is determined based on the original user data of each user dataset.

6. The data collection method as described in claim 4, characterized in that, Adding second identification data whose current occurrence frequency is greater than the occurrence frequency threshold to the user dataset includes: The original identification results, the community-published data, and the current frequency of occurrence of the second identification data are correlated and added to the user dataset.

7. The data collection method as described in claim 4, characterized in that, Adding second identification data whose current occurrence frequency is greater than the occurrence frequency threshold to the user dataset includes: Obtain the data storage information of the original historical data corresponding to the second identification data; Add the data storage information to the user dataset.

8. The data collection method as described in claim 2, characterized in that, The method further includes adding the text recognition results before the preset total dataset: Get the preset set of disabled texts; The preset set of disabled texts is compared with the recognition results of the text recognition. If the recognition result passes the disabled comparison, the passing recognition result will be added to the preset total dataset.

9. The data collection method as described in claim 5, characterized in that, After adding the second identification data whose current occurrence frequency is greater than the occurrence frequency threshold to the user dataset, the method further includes at least one of the following: The user's preference data is determined based on the frequency of occurrence of the original user data in the user dataset, and the user preference data includes at least one original user data. The community preference data is determined based on the frequency of occurrence of existing historical data in the preset total dataset, and the community preference data includes at least one of the existing historical data.

10. A text recognition method, characterized in that, The text recognition method includes: Obtain the user's community data to be identified; The community data to be identified is compared with the user's user dataset in a third comparison. The original user data in the user dataset that is successfully matched in the third comparison is determined as the third identification data. The original user data in the user dataset is collected by the data collection method as described in any one of claims 1-9. The community data to be identified that failed the third comparison is compared with the preset total dataset in the fourth comparison. The original historical data in the preset total dataset that succeeded in the fourth comparison is determined as the fourth identification data. The preset total dataset includes user datasets of multiple users. The text recognition result of the community data to be identified is generated based on the third and fourth identification data.

11. A data collection device, characterized in that, The data collection device includes: The acquisition module is used to acquire data posted by users in the community. The first comparison module is used to perform a first comparison between the community-published data and the user's user dataset, and to determine the original user data in the user dataset that is successfully matched as the first identification data. The user dataset includes multiple original user data and the frequency of occurrence of the original user data. The second comparison module is used to perform a second comparison between the community-published data that failed the first comparison and the preset total dataset, and to determine the original historical data in the preset total dataset that succeeded in the second comparison as the second identification data. The preset total dataset includes multiple original historical data and the frequency of occurrence of the original historical data. The statistics module is used to calculate the current occurrence frequency of the first identification data and the current occurrence frequency of the second identification data based on the occurrence frequency of the original user data and the occurrence frequency of the original historical data. The determination module is used to sort the current occurrence frequency of the first identification data and the current occurrence frequency of the second identification data, determine the current occurrence frequency of the first identification data or the current occurrence frequency of the second identification data with a sorting value of a preset order value as an occurrence frequency threshold, and add the second identification data whose current occurrence frequency is greater than the occurrence frequency threshold to the user dataset to collect the user's user data.

12. An electronic device, characterized in that, The electronic device includes: One or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to perform the method as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by the computer's processor, causes the computer to perform the method according to any one of claims 1 to 10.

Images