Detecting the legitimacy of content related to data collection

Abstract

The present disclosure provides a method and apparatus for detecting the legitimacy of content related to data collection. Content related to data collection can be obtained. At least one identifying element can be detected from the content. An entity corresponding to the identifying element can be identified. The legitimacy of the content can be determined based at least on a correlation between a creator of the content and the entity.

Description

Background Art

[0001] With the development of Internet technology, people can collect data of interest more conveniently through the Internet. Data collection can be carried out in different ways such as forms, web pages, emails, productivity tool documents, etc. In this article, data collection services can broadly refer to various services, applications, software, websites, etc. that can be used to implement data collection or have data collection functions. For example, a survey form service is a dedicated data collection service that collects data through a form. In addition, data collection can also be carried out in services that are not dedicated to data collection, for example, data is collected through web pages in a browser service, data is collected through emails in an email service, data is collected through productivity tool documents in a productivity tool, etc. All these services that can realize data collection can be collectively referred to as data collection services. Summary of the invention

[0002] This Summary is provided to introduce a group of concepts that will be further described in the following Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0003] Embodiments of the present disclosure provide methods and devices for detecting the legitimacy of content related to data collection. Content related to data collection can be obtained. At least one identifying element can be detected from the content. An entity corresponding to the identifying element can be identified. The legitimacy of the content can be determined based at least on the correlation between the creator of the content and the entity.

[0004] It should be noted that one or more of the above aspects include the features specifically pointed out in the following detailed description and claims. The following description and drawings set forth in detail certain illustrative features of the one or more aspects. These features are merely indicative of the various ways in which the principles of various aspects may be implemented, and the present disclosure is intended to include all such aspects and their equivalents. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The disclosed aspects will be described below in conjunction with the accompanying drawings, which are provided to illustrate rather than limit the disclosed aspects.

[0006] Figure 1 An exemplary process for detecting the legitimacy of content related to data collection according to an embodiment of the present disclosure is shown.

[0007] Figure 2 Exemplary real image identifying elements and exemplary non-real image identifying elements obtained by transforming the real image identifying elements are shown.

[0008] Figure 3 Exemplary non-real image identifying elements created for a particular entity are shown.

[0009] Figure 4 An exemplary form for data collection is shown.

[0010] Figure 5 An exemplary process for detecting image identifying elements according to an embodiment of the present disclosure is shown.

[0011] Figure 6 Another exemplary process for detecting image identifying elements according to an embodiment of the present disclosure is shown.

[0012] Figure 7 Another exemplary process for detecting image identifying elements according to an embodiment of the present disclosure is shown.

[0013] Figure 8 An exemplary process for identifying entities corresponding to image identifying elements according to an embodiment of the present disclosure is shown.

[0014] Fig. 9 An exemplary process of generating a training data set for training an entity recognition model based on real identifying elements according to an embodiment of the present disclosure is shown.

[0015] Fig.10 An exemplary splitting of real identifying elements according to an embodiment of the present disclosure is shown.

[0016] Fig.11 An exemplary transformation of a visual component according to an embodiment of the present disclosure is shown.

[0017] Fig.12 is a flowchart of an exemplary method for detecting legitimacy of content related to data collection according to an embodiment of the present disclosure.

[0018] Fig.13 An exemplary device for detecting the legitimacy of content related to data collection according to an embodiment of the present disclosure is shown.

[0019] Fig.14 An exemplary device for detecting the legitimacy of content related to data collection according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0020] The present disclosure will now be discussed with reference to several exemplary embodiments. It should be understood that the discussion of these embodiments is only used to enable those skilled in the art to better understand and thereby implement the embodiments of the present disclosure, and does not teach any limitation on the scope of the present disclosure.

[0021] Data collection services may be used by bad users to collect data for improper purposes, thereby posing the risk of data collection services being abused. For example, data collection services may be maliciously used to collect personal privacy or sensitive data, collect trade secrets, disseminate bad content, etc., and the collected data may be used for financial crimes, reputation infringement, cyber attacks, etc. Data collection for improper purposes will greatly damage the interests of providers and legitimate users of data collection services. Take phishing as an example, which is a common cyber attack. Phishers usually impersonate specific entities to collect data for improper purposes, such as collecting private or sensitive data such as login accounts and passwords, bank card numbers, credit card numbers, home addresses, and company business information. For example, survey form services are data collection services commonly used by phishers. Through survey form services, phishers can create and distribute forms for improper purposes and obtain information provided by respondents.

[0022] Logos are widely used in various data collection services to identify entities associated with the data collection services. In this article, a logo can be an image or text composed of graphics and / or text with a sense of design, such as a legally registered trademark, a widely used icon, etc., and an entity can refer to a brand, product, service, person, etc. targeted by the data collection service. Phishers usually use the logo of a specific entity in a data collection service to deceive users into believing that the data collection service is initiated by the owner or authorized person of the specific entity. In this article, the owner of an entity can refer to a company, organization, individual, etc. that legally owns, possesses and uses the entity, and the authorized person of an entity can refer to a company, organization, individual, etc. that is authorized to use the entity. For example, phishers usually provide a brand logo in an email to make users believe that the email is sent by the owner or authorized person of the brand, thereby further obtaining user data. Existing phishing detection technology can usually detect and identify the entity corresponding to the real logo present in the email, and determine whether the email is a phishing email based on whether the sender of the email is related to the owner of the entity. However, in order to evade detection, phishers often provide some forged logos in emails, such as logos that are different from but similar to real logos, logos newly created for specific entities, etc. Existing phishing detection technologies may have difficulty in identifying such forged logos, and thus cannot determine whether the email is a phishing email. Other data collection services besides emails also face similar problems.

[0023] Embodiments of the present disclosure propose effective detection of the legitimacy of content related to data collection. In this article, content can include various digital information forms that can be used for data collection, such as forms, web pages, emails, productivity tool documents, etc. Correspondingly, data collection services can be various services that support the processing of content, such as survey form services, browser services, email services, productivity tools, etc. In addition, the legitimacy of content can refer to whether the creator of the content is legitimate, or whether the use of specific elements by the content is legitimate, etc. The creator of the content can create content for collecting data in the data collection service, and the responder of the content can fill in information in the content in the data collection service to provide data, wherein the responder can refer to the recipient who has responded to the content among the recipients who received the content. In this article, creators and responders can broadly refer to companies, organizations, individuals, etc. that use data collection services.

[0024] In one aspect, the embodiments of the present disclosure propose to detect an identifying element from content related to data collection, identify an entity corresponding to the identifying element, and determine the legitimacy of the content based on the correlation between the creator of the content and the identified entity. In this article, an identifying element may refer to an element such as a graphic, text, attribute, style, etc. that enables a viewer to associate it with a specific entity. An identifying element may include, for example, a logo, a representative picture, a representative text, etc. A representative picture may be a picture that can represent a specific entity, such as a picture with a slogan for a specific entity, a picture of a building with significant features associated with a specific entity, a head portrait of a person associated with a specific entity, etc. A representative text may be a text that can represent a specific entity, such as the name of a specific entity, a slogan of a specific entity, a description pointing to a specific entity, etc. For example, the description "the world's largest computer software provider" may point to "Microsoft", the description "founder of Microsoft" may point to "Bill Gates", etc. An identifying element usually exists in the content in the form of text or image. In this article, an identifying element in the form of text may be referred to as a text identifying element, and an identifying element in the form of an image may be referred to as an image identifying element.

[0025] In another aspect, the embodiments of the present disclosure propose to separately identify each visual component in an image identifying element, and determine the entity corresponding to the image identifying element based on the identification results of each visual component. In this article, a visual component may refer to an independent and separate part that constitutes an image identifying element. By separately identifying each visual component, it is possible to effectively deal with the behavior of phisher evading detection by changing the relative position of the visual component in the image identifying element.

[0026] In another aspect, an embodiment of the present disclosure proposes to identify visual components through a deep learning-based model, which can be trained using a training data set obtained according to an embodiment of the present disclosure. In addition to including visual component samples from real identification elements, the training data set can also include visual component samples that are different from the visual components in the real identification elements, so that when actually deployed, the trained model can correctly identify a variety of identification elements and is not limited to real identification elements. In this article, real identification elements can refer to identification elements created and / or used by the owner of the entity, and visual component samples that are different from the visual components in the real identification elements can, for example, include visual component samples obtained by transforming the visual components in the real identification elements, visual component samples created for specific entities, and so on.

[0027] In another aspect, the embodiments of the present disclosure propose that when detecting image identification elements from content, a sliding window with a predetermined size is used to scan an image extracted from the content to obtain a group of sub-images, and the image identification elements are detected from the group of sub-images. In this way, even if the image identification element occupies a very small part of the extracted image, it can be detected.

[0028] In another aspect, embodiments of the present disclosure can determine the legitimacy of content related to data collection not only after the content is published, but also before the content is published. For example, the content can be obtained during content creation, and when it is determined that the content is not legitimate, a prompt message can be sent to the creator of the content and / or the data collection service provider, and the prompt message can, for example, warn or remind the creator that the identifying elements are being used improperly in the content. Additionally or alternatively, when it is determined that the content is not legitimate, the content can be prevented from being published, thereby avoiding the improper collection of user data.

[0029] It should be understood that although the foregoing discussion and the following discussion may involve examples of detecting or identifying logos in content, embodiments of the present disclosure are not limited thereto, and any other type of identifying elements may be detected or identified in content in a similar manner.

[0030] Figure 1 An exemplary process 100 for detecting the legitimacy of content related to data collection according to an embodiment of the present disclosure is shown. For example, entities corresponding to identifying elements contained in content related to data collection can be identified, and the legitimacy of the content can be determined based on the correlation between the creator of the content and the identified entity.

[0031] At 102, content related to the data collection may be obtained. For example, corresponding content may be obtained based on the digital information form adopted by the data collection, such as a form, a web page, an email, a productivity tool document, etc. The content may be obtained after the content is published. Alternatively, the content may also be obtained before the content is published, such as during the content creation.

[0032] At 104, at least one identifying element can be detected from the content.

[0033] The text identification element can be directly detected or extracted from the content, or detected or extracted from the information related to the content. The information related to the content can be the code information of the content, such as the HTML code information of the content, the rendering tree (render tree) constructed in the HTML rendering process, etc. In one embodiment, a group of text identification elements can be predefined, and this group of text identification elements can, for example, include representative characters associated with common entities, such as the name of a well-known brand, the name of a famous person, etc. The text identification element can be detected by retrieving the predefined text identification element from the content or from the information related to the content. In addition or alternatively, it can be determined whether it is a text identification element based on whether the position of the text element on the content has identification. For example, a group of positions can be predefined, and this group of positions can, for example, include the position where the text identification element usually appears, such as the upper left area, the upper right area, the lower left area, the lower right area, etc. of the content. It can be determined that the text element is a text identification element based on that the text element is located in one of the predefined positions.

[0034] For image identification elements, images can be extracted from the content, and image identification elements can be obtained from the extracted images. The images can be extracted from code information related to the content, or from the rendering results of the content. Figure 5-Figure 7 The exemplary process of detecting the image identification element is described below. These processes can be respectively applicable to the content where the image identification element is located at a predetermined position and the content where the image identification element is located at an unknown position.

[0035] At 106 , an entity corresponding to the identifying element can be identified.

[0036] For a text identifying element, an entity corresponding to the text identifying element can be identified based on the text content of the text identifying element. For example, for a text identifying element including the text "Microsoft", an entity corresponding thereto can be determined based on the text "Microsoft" to be the brand "Microsoft", which is owned by Microsoft Corporation. In one embodiment, a mapping table including a text identifying element and its corresponding entity can be predefined, and the entity corresponding to the text identifying element can be identified by searching the mapping table or using a mapping rule. In another embodiment, a machine learning model can be utilized to identify the entity corresponding to the text identifying element. The machine learning model can be trained using a training data set including a text identifying element sample and an entity annotation.

[0037] The current machine learning model for identifying entities corresponding to textual identification elements usually relies only on real identification elements for training, so it can only identify real identification elements, and it may be difficult to identify identification elements different from real identification elements. In this article, identification elements different from real identification elements may also be referred to as non-real identification elements, forged identification elements, etc. The embodiment of the present disclosure proposes to improve the training data set for training the machine learning model so that when it is actually deployed, the trained machine learning model can correctly identify real and non-real identification elements and is not limited to real identification elements. For textual identification elements, non-real identification elements can be, for example, identification elements obtained by transforming real identification elements. For example, the non-real text identification element "Microsoft" can be obtained by replacing the letter "o" in the real text identification element "Microsoft" of the brand "Microsoft" with the number "0", and / or replacing the letter "s" with the dollar sign "$". Since "Micro$oft" is close to "Microsoft", the viewer may still associate "Micro$oft" with the brand "Microsoft". According to the training data set for training the machine learning model according to the embodiment of the present disclosure, in addition to including training samples related to real identification elements, it may also include training samples related to non-real text identification elements. The training samples related to non-real text identification elements may include non-real identification element samples and entity annotations corresponding to the non-real identification element samples. The entity annotation may be consistent with the entity annotation of the real identification element sample associated with the non-real identification element sample.

[0038] For an image identifying element, the entity corresponding to the image identifying element can be identified as a whole. Alternatively, candidate entities corresponding to each visual component in the image identifying element can be identified separately, and the entity corresponding to the image identifying element can be determined based on the identification results of the candidate entities. Figure 8 An exemplary process of identifying entities corresponding to identifying elements of an image is described below.

[0039] Visual components may include textual visual components and non-textual visual components. In this article, textual visual components may refer to visual components that only present text, and non-textual visual components may refer to visual components that at least present graphics. For example, entities corresponding to textual visual components may be identified by OCR, and entities corresponding to non-textual visual components may be identified at least by a model based on deep learning. In this article, a model for identifying entities corresponding to non-textual visual components may be referred to as an entity recognition model. In addition, the entity recognition model may also identify textual visual components that are difficult to identify by OCR, such as textual visual components containing texts using artistic fonts, textual visual components containing deformed texts, and the like. Current entity recognition models are usually trained only by relying on real identifying elements, so that they can only identify real identifying elements, and may be difficult to identify identifying elements different from real identifying elements. For image identifying elements, non-real identifying elements may, for example, include identifying elements obtained by transforming real identifying elements, identifying elements created for specific entities, and the like. The embodiments of the present disclosure propose to improve the training data set used to train the entity recognition model so that when it is actually deployed, the trained entity recognition model can correctly identify real and non-real image identification elements instead of being limited to real image identification elements. Fig. 9 To illustrate an exemplary process of generating a training data set for training an entity recognition model.

[0040] The non-real image identification element may be, for example, an identification element obtained by transforming a real identification element. Figure 2Exemplary real image identification element and exemplary non-real image identification element obtained by transforming real image identification element are shown. Identification element 200a can be, for example, the real sign of brand "AABB", which can include the non-text visual component consisting of rectangle, circle and triangle and the text visual component consisting of text "AABB", and the text visual component is located on the right side of the non-text visual component. Identification element 200b-200d can be, for example, the non-real sign obtained by transforming the real sign. For example, identification element 200b also includes the non-text visual component and the text visual component identical with identification element 200a, but in identification element 200b, the text visual component is located below the non-text visual component rather than on the right side. The relative position between the visual components in identification element 200c is identical with identification element 200a, but the non-text visual component in identification element 200c is rotated relative to the non-text visual component in identification element 200a. The relative positions of the visual components in the identification element 200d are the same as those in the identification element 200a, but the non-text visual components in the identification element 200d are flipped relative to the non-text visual components in the identification element 200a.

[0041] It should be understood that Figure 2 The illustrated identification elements 200b-200d are merely several examples of non-real image identification elements obtained by transforming the real identification elements of the image of a specific entity. The non-real image identification elements obtained by transforming the real image identification elements may also have any other forms, such as having a different color from the real image identification elements, having an artistic font different from the real image identification elements, etc.

[0042] A non-real image identifying element may also be, for example, an image identifying element created for a specific entity. Figure 3 2 shows an exemplary non-real image identification element created for a specific entity. The identification element 300a and the identification element 300b may be created for the brand "AABB", for example, which may not be created by targeting a specific entity. Figure 2 However, since the logo element 300a and the logo element 300b contain the text "AABB", the viewer is very likely to mistake them for the logo of the brand "AABB".

[0043] It should be understood that Figure 3The illustrated image identification elements are only some examples of non-real image identification elements created for a specific entity. The non-real image identification elements created for a specific entity may also have any other forms, such as pictures with the advertising slogan of the specific entity, photos of the stores of the specific entity, etc.

[0044] After identifying the entity corresponding to the identifying element, the legitimacy of the content can be determined based at least on the correlation between the creator of the content and the identified entity. At 108, it can be evaluated whether the creator of the content is related to the identified entity. In one embodiment, if the creator of the content is the same as the owner of the entity, the creator of the content can be considered to be related to the entity. For example, when the logo detected from the form corresponds to the brand "AABB", it can be determined whether the creator of the form is the same as the owner of the brand "AABB". If the same, the creator of the form can be considered to be related to the brand "AABB". In another embodiment, if the creator of the content is authorized by the owner of the entity to use the identifying element of the entity, the creator of the content can be considered to be related to the entity. It can be determined, for example, by viewing the permission information of the creator of the content to determine whether it has such authorization. Alternatively, it can be determined, for example, by viewing the authorization information of the entity to determine whether it authorizes the creator to use its identifying element. It should be understood that the above criteria for evaluating the correlation between the creator of the content and the entity are only exemplary, and the correlation between the creator of the content and the entity can also be evaluated by other criteria. For example, if the creator of the content is an employee of the entity's owner, then the creator of the content may be considered to be associated with the entity. For example, if the domain name of the verified email address of the creator of the content matches the domain name of the email address of the entity's owner, then the creator of the content may be considered to be associated with the entity.

[0045] If it is assessed at 108 that the creator of the content is related to the identified entity, process 100 may proceed to 110, where it is determined that the content is legitimate. If it is assessed at 108 that the creator of the content is not related to the identified entity, process 100 may proceed to 112, where it is determined that the content is illegitimate. When the content is determined to be illegitimate, some measures may be taken to control or intervene in it, such as prohibiting the data collection, sending a reminder to the user, and the like.

[0046] It should be understood that Figure 1The process 100 in is merely an example of a process for detecting the legitimacy of content associated with data collection. Depending on the actual application requirements, the process for detecting the legitimacy of content associated with data collection may include any other steps, and may include more or fewer steps. In addition, the evaluation result of the relevance between the creator of the content and the entity made at 108 is only one of the factors used to determine the legitimacy of the content. The determination of the legitimacy of the content may also be based on other factors, such as the type of data collected by the content, the purpose of the data collection involved in the content, and the like.

[0047] In addition, it should be understood that although the foregoing discussion and the following discussion may involve detecting one identifying element from given content, the embodiments of the present disclosure are not limited to this. It is also possible to detect multiple identifying elements from given content. According to the legitimacy detection process of the embodiments of the present disclosure, all detected identifying elements can be processed to identify the entities corresponding to each identifying element. The identified entities may be the same or different. In the case where the identified entities are different, the legitimacy of the content can be determined based on the correlation between the creator of the content and the various entities.

[0048] It should also be understood that although the foregoing discussion and the following discussion may involve identifying an entity corresponding to a single identifying element as a unique entity, the embodiments of the present disclosure are not limited to this. In some cases, such as when there are multiple similar identifying elements for different entities, the entity corresponding to a single identifying element may be identified as two or more entities, and the confidence scores associated with each entity indicating the trustworthiness of the corresponding entity may also be relatively close. In this case, the legitimacy of the content can be determined based on the correlation between the creator of the content and the various entities.

[0049] According to an embodiment of the present disclosure, process 100 may be performed before content related to data collection is published. For example, the content may be obtained during content creation. When it is determined that the content is not legitimate, a prompt message may be sent to the creator of the content, which may, for example, warn or remind the creator that an identifying element may be used improperly in the content. Additionally or alternatively, when it is determined that the content is not legitimate, the content may be prevented from being published, thereby preventing the user data from being improperly collected.

[0050] Figure 4An exemplary form 400 for data collection is shown. The title "Account Manager" of the form 400 indicates that the form is intended to assist the recipient in logging into his account. A logo 402 is provided in the upper left corner of the form 400, which is intended to identify the entity associated with the data collection service. The creator of the form creates questions in the form for collecting user data, such as "Enter your username", "Enter your password", etc. If the recipient gives answers to these questions, the creator obtains the desired user data. It should be understood that Figure 4 This is just one example of data collection, and in actual scenarios there may be various other forms of data collection.

[0051] The process for detecting the legitimacy of content related to data collection according to an embodiment of the present disclosure may be adopted, for example Figure 1 100 in the process to detect the legitimacy of the form. For example, the form 400 can be obtained and an identifying element, such as a logo 402, can be detected from the form 400. Subsequently, an entity corresponding to the logo 402, such as the brand "AABB", can be identified. Although the logo 402 is different from the real logo of the brand "AABB", for example Figure 2 , but according to the entity recognition process of the embodiment of the present disclosure, for example Figure 1 In step 106, it can still be identified that the entity corresponding to the logo 402 is the brand "AABB". Next, it can be evaluated whether the creator of form 400 is related to the brand "AABB". For example, if the creator of form 400 is the same as the owner of the brand "AABB", or the creator of form 400 is authorized by the owner of the brand "AABB" to use its logo, it can be considered that the creator of form 400 is related to the brand "AABB". Otherwise, it can be considered that the creator of form 400 is unrelated to the brand "AABB". When the creator of form 400 is related to the brand "AABB", form 400 can be considered legitimate; and when the creator of form 400 is unrelated to the brand "AABB", form 400 can be considered illegitimate.

[0052] According to an embodiment of the present disclosure, when detecting image identification elements from content, an image may be first extracted from the content, and then the image identification elements may be obtained from the extracted image. Figure 5-Figure 7 Exemplary processes 500-700 for detecting image identifying elements are shown. These processes can be respectively applicable to content where the image identifying element is located at a predetermined position and content where the image identifying element is located at an unknown position.

[0053] Figure 5An exemplary process 500 for detecting an image identifying element according to an embodiment of the present disclosure is shown. The process 500 may be applicable to content where an image identifying element is located at a predetermined position. For example, for a form, the image identifying element is usually located at the upper left area of ​​the form.

[0054] At 510, an image can be extracted from a predetermined position of the content. Taking a form as an example, the image identification element is usually located at, for example, the upper left area of ​​the form. Accordingly, an image containing the image identification element is also usually located in this area, so the image can be extracted from this area.

[0055] Image identifying elements can be obtained from the extracted image. Image identifying elements can be obtained, for example, by a model based on deep learning. In this article, the model for obtaining the image identifying elements at a predetermined position can be referred to as an identifying element recognition model. The identifying element recognition model can obtain the image identifying elements without determining the position of the image identifying elements. The identifying element recognition model usually requires that the input image has a specified size. At 520, the image extracted from the content can be standardized to the specified size required by the identifying element recognition model.

[0056] At 530 , image identifying elements may be obtained from the normalized image by a identifying element recognition model.

[0057] Although in some content such as forms, images containing identifying elements are usually located at a predetermined position and their size is within a predetermined range, phishers can still take some evasive actions to prevent the image identifying elements from being obtained from the image. For example, the image identifying elements can be placed at different positions in the image, or the image identifying elements can occupy a small part of the image.

[0058] In order to deal with this type of evasive behavior by phishers, an embodiment of the present disclosure proposes to improve the training data set for training the identification element recognition model. An image sample set including multiple image samples can be generated as a training data set for training the identification element recognition model. Each image sample in the image sample set can contain an identification element sample. The identification element sample can be located at any position in the image sample. In addition, the size of the identification element sample and the ratio of the size of the image sample in which it is located can be within a predetermined range, such as 20% to 100%. When the identification element recognition model trained by such an image sample set is actually deployed, it can effectively obtain identification elements from various types of images, such as images in which the identification element is located at any position thereof, images in which the size of the identification element and the size of the image are within a predetermined range, and the like.

[0059] Figure 6Another exemplary process 600 for detecting image identifying elements according to an embodiment of the present disclosure is shown. Process 600 can be applicable to content where the identifying elements are located at unknown locations. For example, in content such as web pages, emails, productivity tool documents, the location of the image identifying elements is usually uncertain or unknown. For such content, an object detection method can be used to detect the image identifying elements. For example, an image can be extracted from the content, and the extracted image can be used as a whole to detect the image identifying elements therefrom.

[0060] At 610, an image may be extracted from the content. Since the position of the image identifying element is unknown, images at various positions in the content may be extracted.

[0061] Image identifying elements can be obtained from the extracted image. Image identifying elements can be obtained, for example, by a model based on deep learning. In this article, the model for obtaining image identifying elements at unknown positions can be referred to as an identifying element detection model. The identifying element detection model can first determine the position of the image identifying element in the image, and then obtain the image identifying element from the position. The identifying element detection model usually requires that the input image has a specified size. At 620, the image extracted from the content can be standardized to the specified size required by the model.

[0062] At 630 , the location of the image identifying element in the extracted image may be determined by the identifying element detection model.

[0063] At 640 , the image identifying element may be acquired from the determined position by the identifying element detection model.

[0064] An image sample set including a plurality of image samples can be used to train an identifying element detection model. Each image sample in the image sample set can, for example, contain an identifying element sample and the position of the identifying element sample in the image sample. When the identifying element detection model trained using such an image sample set is actually deployed, the position of the image identifying element can be determined from the input image, and the image identifying element can be acquired from the determined position. In addition, the identifying element recognition model for acquiring the image identifying element at a predetermined position and the identifying element detection model for acquiring the image identifying element at an unknown position can be collectively referred to as an identifying element acquisition model.

[0065] In addition, for content where the image identifying element is located at an unknown position, phishers may place the image identifying element in a larger image, which may be, for example, a background image of a web page or an email, and the image identifying element only occupies a very small part of the image. In this case, since the model used to obtain the image identifying element from the image, such as the aforementioned identifying element detection model, usually requires the input image to have a specified size, when the image containing the image identifying element is standardized to the specified size, the image identifying element will become too small to be acquired by the model. In order to deal with such circumvention behavior, an embodiment of the present disclosure proposes to obtain a group of sub-images by scanning the image using a sliding window, and detect the identifying element from the group of sub-images. Figure 7 Another exemplary process 700 for detecting image identifying elements according to an embodiment of the present disclosure is shown. The process 700 may be applicable to content where the identifying element is located at an unknown position and the image identifying element occupies a very small portion of the image.

[0066] At 710, an image may be extracted from the content. Since the position of the image identifying element is unknown, images at various positions in the content may be extracted.

[0067] For each image in the extracted image, at 720, a set of sub-images can be obtained by scanning the image using a sliding window. The size of each sub-image can be consistent with the size of the sliding window. The size of the sliding window can be set to be larger than the size of the image identification element so that it can include the image identification element. Since the size of the image extracted from the content is known, in order to enable the recipient of the content to notice the image identification element contained therein, the size of the image identification element should also be within a reasonable range, so the size of the image identification element can be roughly estimated based on the size of the image. In addition, when the size of the image extracted from the content is certain, in order to make the number of sub-images as small as possible to improve efficiency, the size of the sub-image should be as large as possible. However, in order to enable the image identification element to be obtained from the sub-image, the ratio of the size of the image identification element to the size of the sub-image should be higher than a predetermined threshold. That is, the size of the sub-image, that is, the size of the sliding window, can be smaller than the size of the image identification element by a predetermined multiple. Therefore, the size of the sliding window can be larger than the size of the image identification element and smaller than the size of the image identification element by a predetermined multiple.

[0068] In addition, in order to allow the entire image identification element to be contained in a single sub-image, the group of sub-images may also have a predetermined overlap. The predetermined overlap may be set to 50%, for example. In this case, the sliding window may be moved horizontally or vertically by 50% of its side length each time, so that the newly obtained sub-image has a 50% overlap with the previous sub-image.

[0069] The image identifying element can be obtained from the obtained group of sub-images. Since the ratio of the size of the image identifying element to the size of the sub-image is within a predetermined range, the image identifying element can be obtained from the sub-image, for example, by an identifying element recognition model. The identifying element recognition model can be, for example, the identifying element recognition model trained using the image sample set described above. The identifying element recognition model usually requires that the input image has a specified size. At 730, each sub-image in the group of sub-images can be standardized to the specified size required by the identifying element recognition model.

[0070] At 740 , image identifying elements may be obtained from the normalized set of sub-images, for example, by a identifying element recognition model.

[0071] It should be understood that the above Figure 5-Figure 7 The process described for detecting image identification elements is only exemplary. Depending on actual application requirements, the process for detecting image identification elements may include any other steps, and may include more or fewer steps. Figure 7 In the process, before scanning the image extracted from the content using the sliding window, it can be determined whether the size of the image exceeds a predetermined threshold, and scanning is performed if the size exceeds the threshold. Otherwise, the image can be directly standardized to the specified size required by the identification element detection model and the image identification element can be obtained from the standardized image, such as Figure 6 As described in steps 620-640 in . In addition, when there may be multiple image identification elements of different sizes in the image, multiple sliding windows with corresponding sizes can be used to scan the image to obtain multiple groups of sub-images, and corresponding image identification elements can be obtained from each group of sub-images. In addition, although processes 500-700 include standardizing the image or sub-image and obtaining identification elements from the standardized image or sub-image, in some cases, the standardization processing can be omitted from processes 500-700. For example, when the size of the image or sub-image is lower than a predetermined threshold, the identification elements can be directly obtained from the image or sub-image without standardizing it.

[0072] Return to reference Figure 2, which shows a real identifying element and several non-real identifying elements. For example, identifying element 200a can be a real identifying element including a non-text visual component composed of a square, a circle and a triangle and a text visual component formed by the text "AABB". Identifying element 200b includes the same non-text visual component and text visual component as identifying element 200a, but its text visual component is located below the non-text visual component, rather than to the right. Phishers often use identifying elements whose relative positions of visual components are different from those of real identifying elements to evade identification.

[0073] In order to deal with such circumvention behavior, an embodiment of the present disclosure proposes to separately identify each visual component in the image's identifying element, and determine the entity corresponding to the identifying element based on the identification results of each visual component. Figure 8 An exemplary process 800 for identifying an entity corresponding to an identifying element according to an embodiment of the present disclosure is shown. Process 800 can be performed for an image identifying element having one or more visual components. Candidate entities corresponding to each visual component in the image identifying element can be identified, and the entity corresponding to the image identifying element can be determined based at least on the candidate entity. Optionally, the position of the visual component can also be determined, and the entity corresponding to the image identifying element can be determined based on both the candidate entity and the position.

[0074] At 810, candidate entities corresponding to the various visual components in the image identifying elements may be identified respectively. According to an embodiment of the present disclosure, for example, the text visual component may be identified by OCR, and at least the non-text visual component may be identified by an entity recognition model based on deep learning. In addition, the entity recognition model may also identify text visual components that are difficult for OCR to identify, such as text visual components containing text using artistic fonts, text visual components containing deformed text, and the like. Figure 2 Taking the identifying element 200b in the example, the first candidate entity set corresponding to the non-text visual component above the identifying element 200b can be identified by the entity recognition model, and the second candidate entity set corresponding to the text visual component below it can be identified by OCR. A single visual component may correspond to one or more candidate entities. For example, the first candidate entity set may include the brand "AABB" and the brand "CCDD", and the second candidate entity set may include the brand "AABB".

[0075] Optionally, the positions of the various visual components may be determined at 820. For example, the non-text visual component at the top of the identifying element 200b may be determined to be at a first position, and the text visual component at the bottom thereof may be determined to be at a second position.

[0076] Optionally, at 830, the distance between each two visual components can be calculated based on the determined positions of the respective visual components to obtain a set of distances. For example, the distance between the two visual components can be calculated based on the first position of the non-text visual component of the identifying element 200b and the second position of the text visual component.

[0077] At 840, an entity corresponding to the identifying element may be determined based at least on the candidate entities identified at 810. Continuing with the example of the identifying element 200b, the first candidate entity set includes the brand "AABB" and the brand "CCDD" and the second candidate entity set includes the brand "AABB", and accordingly, the entity corresponding to the identifying element 200b may be determined to be the brand "AABB".

[0078] Optionally, the entity corresponding to the identity may be further determined based on a set of distances calculated at 830 .

[0079] In the case where this set of distances includes only one distance, that is, the identifying element includes only two visual components, if the distance is lower than a predetermined threshold, the two visual components may belong to one identifying element. Therefore, the entity corresponding to the identifying element can be determined based on two candidate entity sets corresponding to the two visual components. For example, for the identifying element 200b, it can be determined whether the distance between its two visual components is lower than the threshold. If the distance is lower than the threshold, the entity corresponding to the identifying element 200b can be determined based on the first candidate entity set including the brand "AABB" and the brand "CCDD" and the second candidate entity set including the brand "AABB". The entity can be determined as the brand "AABB", for example. If the distance is not lower than the predetermined threshold, the two visual components may not belong to one identifying element. Therefore, two candidate entity sets corresponding to the two visual components can be directly provided as entity recognition results. Continuing to take the identifying element 200b as an example, if the distance is not lower than the threshold, the brand "AABB" and the brand "CCDD" are directly provided as the recognition results of the non-text visual components, and the brand "AABB" is provided as the recognition result of the text visual component.

[0080] In the case where this set of distances includes multiple distances, that is, the identifying element includes more than two visual components, if the distance between each visual component of all visual components and its closest visual component is lower than a threshold value, then all visual components may belong to one identifying element. Therefore, the entity corresponding to the identifying element can be determined based on the candidate entities corresponding to each visual component. If the distance between one or more visual components and their closest visual components is not lower than a threshold value, then the one or more visual components may not belong to the identifying element. In this case, the candidate entities corresponding to the one or more visual components may not be considered when determining the entity corresponding to the identifying element.

[0081] In the above manner, even if the relative position of the visual component of the identifying element is different from that of the real identifying element, the entity corresponding to the identifying element can be correctly identified.

[0082] Combination of the above Figure 1 The process described for identifying entities corresponding to textual identifying elements only considers the textual identifying elements themselves, and the above combined Figure 8 The process 800 described for identifying the entity corresponding to the image identifying element also only considers the image identifying element itself. In order to more accurately identify the entity corresponding to the text identifying element or the image identifying element, the embodiment of the present disclosure proposes that entity recognition can also be performed based on the context of the text identifying element or the image identifying element. In this article, the context can refer to the content where the identifying element is located, such as a form, a web page, an email, a productivity tool document, etc. In one embodiment, the entity corresponding to the identifying element can be identified at least based on the context of the identifying element. For the text identifying element, the context of the text identifying element and the text identifying element can be provided to a model for identifying the entity corresponding to the text identifying element. The model can identify the entity corresponding to the text identifying element at least based on the context. For the image identifying element, the context of the image identifying element and the image identifying element can be provided to the entity recognition model. The entity recognition model can identify the candidate entity corresponding to each visual component in the image identifying element at least based on the context. Subsequently, the entity corresponding to the image identifying element can be determined, for example, by steps similar to steps 820-840 in process 800. Alternatively, the entity corresponding to the identifying element may be identified based at least on a portion of the context of the textual identifying element or the image identifying element, rather than the entire context. For example, an area of ​​a predetermined size including the identifying element may be intercepted from the context, and the intercepted area may be provided to the corresponding model together with the identifying element.

[0083] As described above, non-text visual components and text visual components that are difficult to identify by OCR can be identified by an entity recognition model. The entity recognition model can be trained using a training data set according to an embodiment of the present disclosure. The training data set can be a visual component sample set including multiple visual component samples. In addition to including visual component samples from real identifying elements, the visual component sample set can also include visual component samples that are different from the visual components in the real identifying elements, so that when actually deployed, the trained model can correctly identify real and non-real identifying elements and is not limited to real identifying elements. Visual component samples that are different from the visual components in the real identifying elements can be generated based on real iconic elements, or created for specific entities.

[0084] Fig. 9 An exemplary process 900 for generating a training data set for training an entity recognition model based on real iconic elements according to an embodiment of the present disclosure is shown.

[0085] At 910, real identifying elements can be collected. For example, they can be collected from the Internet and accessible phishing content. The entity corresponding to each real identifying element can be determined, and the determined entity is assigned to the real identifying element as an entity annotation. In addition, the context in which the real identifying element is located can also be collected, such as a form, a web page, an email, a productivity tool document, etc. Optionally, the position of each real identifying element can also be determined, such as its position in the content, and the determined position is assigned to the real identifying element as a position annotation.

[0086] At 920 , the collected real identifying elements can be analyzed and each real identifying element can be split into one or more visual components. Fig.10 An exemplary splitting of real identifying elements according to an embodiment of the present disclosure is shown. Fig.10 It can be, for example, a real identifying element of the brand "AABB", which can correspond to Figure 2 The real identifying element 200a can be split into a visual component 1000a consisting of a rectangle, a circle and a triangle and a visual component 1000b consisting of the text "AABB".

[0087] At 930 , visual component samples may be obtained by transforming the separated visual components of the real identifying element. Fig.11 An exemplary transformation of a visual component according to an embodiment of the present disclosure is shown.

[0088] In one embodiment, the visual component may be rotated. For example, Fig.11The visual component sample 1100a in can be obtained by Fig.10 The visual component 1000a is obtained by rotating 180 degrees clockwise.

[0089] In another embodiment, the visual component can be flipped. For example, Fig.11 The visual component sample 1100b in can be obtained by Fig.10 It is obtained by flipping the visual component 1000a in.

[0090] In another embodiment, the color of the visual component can be changed to other colors. For example, the color of at least a part of the visual component can be changed to some classic colors, such as red, yellow, black, etc. In addition, the color of each part of the visual component can be rotated, symmetrically changed, etc. Assume that Fig.10 In the visual component 1000a in , the rectangle is red, the circle is yellow and the triangle is brown. The colors of these parts can be rotated to change, for example, the circle is changed to red, the triangle is changed to yellow, and the rectangle is changed to brown, etc.

[0091] In yet another embodiment, the color of the visual component can be converted to grayscale. Colors are ever-changing, and phishers can easily change the color of the visual component or the identifying element to a color that the entity recognition model has never seen. In order to cope with a variety of color changes, the embodiment of the present disclosure proposes to convert the color of the visual component samples used to train the entity recognition model into grayscale to encourage the model to focus on recognizing the shape of the visual component rather than the color. For example, Fig.11 The visual component sample 1100c in can be obtained by Fig.10 The visual component 1000a in the sample 1100c is obtained by converting the color of the visual component 1000a into grayscale. The various parts in the visual component sample 1100c may have different shades of grayscale, which may correspond to different colors in the original visual component, ie, the visual component 1000a.

[0092] In addition, the context can also be transformed. For example, some information in the context, such as template, color, etc., can be changed.

[0093] It should be understood that the above lists only some exemplary transformations of visual components and contexts, and the embodiments of the present disclosure are not limited thereto, and other transformations of visual components and contexts may also be made according to actual application requirements.

[0094] After transforming the visual components, an extended visual component sample set may be obtained. The extended visual component sample set may include the original visual components extracted from the real identifying elements. In addition, after transforming the context, an extended context sample set may be obtained. The extended context sample set may include the original context.

[0095] At 940, each visual component sample in the extended visual component sample set can be combined with different context samples in the extended context sample set to generate multiple training samples, each training sample comprising a visual component sample and a context sample. During the combination, one or more data enhancement operations, such as rotation, adding noise, perspective transformation, displacement, etc., can also be applied to the visual component sample. Each training sample can include an entity annotation, which can be consistent with the entity annotation of the real identifying element associated with the visual component sample. Optionally, each training sample can also include a position annotation, which can correspond to the position annotation of the real identifying element associated with the visual component sample.

[0096] It should be understood that Fig. 9 The process 900 in is only an example of the process of generating a training data set for training an entity recognition model based on real iconic elements. According to actual application requirements, the process for generating a training data set may include any other steps, and may include more or fewer steps. For example, although the process 900 includes splitting the real identifying elements into visual components, and accordingly, the generated training samples include visual component samples, it is also feasible not to split the real identifying elements. For example, multiple identifying element samples can be generated by directly transforming the real identifying elements, such as the transformation at step 930. Each identifying element sample can be combined with a context sample to generate a training sample. The entity recognition model trained with such training samples can identify the image identifying elements as a whole when deployed. In addition, although the process 900 includes the collection and transformation of context, it is also feasible to only collect the real identifying elements and only transform the visual components in the real identifying elements. In this case, the training sample may not include the context sample. Accordingly, the entity recognition model trained with such training data set may not consider the context of the identifying elements when performing entity recognition.

[0097] Combination of the above Fig. 9 Describes the process of generating a set of training data for training entity recognition models based on real iconic elements. Phishers may also use newly created iconic elements, such as Figure 3In order to enable the entity recognition model to recognize such newly created identifying elements, an embodiment of the present disclosure proposes to add some iconic element samples or visual component samples created for specific entities to the training data set used to train the entity recognition model. For example, such iconic element samples or visual component samples may be actively created by a party providing phishing detection services for some well-known brands, products, etc. Alternatively, iconic element samples or visual component samples created by phishers may also be collected from reported phishing content.

[0098] The training data set generated by the above method can be used to train the entity recognition model. The training data set may include or not include position annotations. The training data set including position annotations can be used to train the entity recognition model for processing the content where the image identification element is located at an unknown position. The training data set not including position annotations can be used to train the entity recognition model for the content where the image identification element is located at a predetermined position.

[0099] In the training data set generation process as described above, the visual components of the image identifying elements are regarded as separate training samples. Accordingly, when the entity recognition model trained with such training samples recognizes the identifying elements, it also recognizes the individual visual components in the image identifying elements separately. In this way, it is possible to effectively deal with the behavior of phisher evading detection by changing the relative positions of the visual components in the image identifying elements. In addition, the training data set generated by the above-mentioned method can have a plurality of different training samples for each entity, such as visual component samples extracted from real identifying elements, visual component samples obtained by transforming the visual components in the real identifying elements, and visual component samples created for specific entities. These training samples enrich the training data set, so that the trained entity recognition model can correctly recognize a variety of image identifying elements when it is actually deployed, and is not limited to real identifying elements.

[0100] In addition, the training samples for each entity in the training data set generated by the above method can be randomly divided into three subsets: a training subset for adapting model parameters using supervised learning, a validation subset for adjusting the model's hyperparameters, selecting the best model and determining the termination point during model training, and a test subset for evaluating the performance of the final model.

[0101] It is described above that a plurality of visual component samples with different colors can be generated by changing the color of the visual component into other colors. A training data set including a plurality of visual component samples with different colors can be used to train an entity recognition model. The trained entity recognition model can be referred to as a color model. In addition, a grayscale visual component sample can be generated by converting the color of the visual component into grayscale. The entity recognition model can be trained using a training data set including grayscale component samples. The trained entity recognition model can be referred to as a grayscale model. When identifying an entity corresponding to a visual component, identification can be performed using a color model and a grayscale model respectively, and the results of identification by each model can be combined into an identification result for the visual component.

[0102] There may also be other elements in content such as forms, web pages, emails, and productivity tool documents that enable the recipient of the content to associate it with a specific entity, such as templates, styles, colors, etc. that the specific entity is accustomed to using. Phishers may use such elements in their content to encourage the recipient of the content to believe that the content is from a specific entity corresponding to the element. An embodiment of the present disclosure proposes to identify entities corresponding to other elements. In this article, the identified entities corresponding to other elements may be referred to as reference entities. For example, image processing technology can be used to extract templates or styles from given content, and recognition or pattern matching can be used to identify reference entities corresponding to the extracted templates or styles. Then, the final entity can be determined based on the reference entity and the entity corresponding to the identifying element in the content. In this way, the entity can be determined more accurately. For example, in the case where the entities corresponding to the identifying element are two or more entities, and the reference entity is one of the two or more entities, the final entity can be determined as the reference entity. Subsequently, the legitimacy of the content can be determined based at least on the correlation between the creator of the content and the final entity. For example, by comparing with Figure 1 The legitimacy of the content is determined in a manner similar to steps 108-110 in FIG.

[0103] It should be understood that although in the above description, obtaining the image identifying elements and identifying the entities corresponding to the image identifying elements are performed by separate models, for example, obtaining the image identifying elements is performed by the identifying element acquisition model, and identifying the entities corresponding to the image identifying elements is performed by the entity recognition model, but the embodiments of the present disclosure are not limited to this. For example, the identifying element acquisition model and the entity recognition model can be integrated into an integrated model. The integrated model can detect the identifying elements from the input content and identify the entities corresponding to the identifying elements. Some neural networks can be trained using the improved training data set according to the embodiments of the present disclosure to obtain a suitable integrated model. For content where the identifying elements are located at predetermined positions, neural networks such as Residual Networks (ResNet) and EfficientNet can be used for training. For content where the identifying elements are located at unknown positions, neural networks such as EfficientDet, Single Shot MultiBox Detector (SSD), and Faster-Region Convolutional Neural Network (Faster-RCNN) can be used for training.

[0104] According to an embodiment of the present disclosure, when selecting a neural network for training, the corresponding computing resource conditions, such as CPU resources, memory resources, etc., can be used to achieve reasonable use of computing resources while ensuring that accuracy and latency requirements are met. For example, compared with ResNet, EfficientNet has a simpler network structure and uses fewer computing resources, but can still achieve higher accuracy. Therefore, when computing resources are limited, EfficientNet can be used; and when computing resources are sufficient, ResNet can be used.

[0105] Fig.12 is a flow chart of an exemplary method 1200 for detecting legitimacy of content related to data collection according to an embodiment of the present disclosure.

[0106] At step 1210, content related to data collection may be obtained.

[0107] At step 1220, at least one identifying element may be detected from the content.

[0108] At step 1230, an entity corresponding to the identifying element may be identified.

[0109] At step 1240, the legitimacy of the content may be determined based at least on the relevance between the creator of the content and the entity.

[0110] In one implementation, the content may be obtained during creation of the content.

[0111] The method 1200 may further include, in response to determining that the content is improper, performing at least one of the following: sending a prompt message to the creator and / or data collection service provider; and preventing the content from being published.

[0112] In one embodiment, the identifying element is different from an actual identifying element of the entity.

[0113] In one implementation, the identification element may include a text identification element. The text identification element may be detected from the content or information related to the content.

[0114] In one embodiment, the identification element may include an image identification element. Detecting at least one identification element may include: extracting an image from the content; and acquiring the image identification element from the image.

[0115] The extracting the image may include extracting the image from a predetermined position of the content.

[0116] The acquiring the image identifying element may include: determining a position of the image identifying element in the image; and acquiring the image identifying element from the position.

[0117] The acquiring the image identification element may include: obtaining a group of sub-images by scanning the image using a sliding window; and acquiring the image identification element from the group of sub-images.

[0118] The size of the sliding window may be larger than the size of the image identifying element and smaller than a predetermined multiple of the size of the image identifying element. The size of the image identifying element may be estimated based on the size of the image.

[0119] The acquisition of the image identifying element may be performed by an identifying element acquisition model. The identifying element acquisition model may be a deep learning-based model trained using an image sample set. Each image sample in the image sample set may contain an identifying element sample.

[0120] The identification element sample may be located at any position in the image sample. The ratio of the size of the identification element sample to the size of the image sample may be within a predetermined range.

[0121] In one embodiment, the identifying element may include an image identifying element. The identifying entity may include: identifying at least one candidate entity corresponding to at least one visual component in the image identifying element; and determining the entity based at least on the at least one candidate entity.

[0122] The method 1200 may further include determining at least one position of the at least one visual component, and wherein the entity is further determined based on the at least one position.

[0123] The at least one visual component may include a textual visual component.The at least one candidate entity may be identified by OCR.

[0124] The at least one visual component may include a non-textual visual component.The at least one candidate entity may be identified by an entity recognition model.

[0125] The entity recognition model may be a deep learning based model trained using a visual component sample set. Each visual component sample in the visual component sample set may have an entity annotation.

[0126] The visual component samples in the visual component sample set may be obtained by at least one of the following ways: extracting from real identifying elements; transforming the visual components in the real identifying elements; and creating for a specific entity.

[0127] The transforming of the visual component may include at least one of: rotating the visual component; flipping the visual component; changing the color of the visual component to another color; and converting the color of the visual component to grayscale.

[0128] In one implementation, the entity may be identifiable based at least on the context of the identifying element.

[0129] In one embodiment, method 1200 may also include: identifying a reference entity corresponding to other elements in the content, the other elements including at least one of a template, a style, and a color; and determining a final entity based on the entity corresponding to the identifying element and the reference entity, and wherein the legitimacy of the content may be determined based at least on the relevance between the creator and the final entity.

[0130] In one embodiment, the content may include at least one of a form, a web page, an email, and a productivity tool document. The identifying element may include at least one of a logo, a representative picture, and a representative text. The entity may include at least one of a brand, a product, a service, and a person.

[0131] It should be understood that method 1200 may also include any steps / processes for detecting the legitimacy of content related to data collection according to the above-mentioned embodiments of the present disclosure.

[0132] Fig.13 An exemplary device 1300 for detecting legitimacy of content related to data collection according to an embodiment of the present disclosure is shown.

[0133] The device 1300 may include: a content acquisition module 1310, used to obtain content related to data collection; an identification element detection module 1320, used to detect at least one identification element from the content; an entity recognition module 1330, used to identify an entity corresponding to the identification element; and a legitimacy determination module 1340, used to determine the legitimacy of the content based at least on the correlation between the creator of the content and the entity.

[0134] It should be understood that the device 1300 may also include any other modules configured to detect the legitimacy of content related to data collection according to the above-mentioned embodiments of the present disclosure.

[0135] Fig.14 An exemplary device 1400 for detecting legitimacy of content related to data collection according to an embodiment of the present disclosure is shown.

[0136] The device 1400 may include at least one processor 1410. The device 1400 may also include a memory 1420 connected to the processor 1410. The memory 1420 may store computer executable instructions, which, when executed, cause the processor 1410 to perform any operation of the method for detecting the legitimacy of content related to data collection according to the above-mentioned embodiments of the present disclosure.

[0137] Embodiments of the present disclosure may be embodied in a non-transitory computer-readable medium. The non-transitory computer-readable medium may include instructions that, when executed, cause one or more processors to perform any operation of the method for detecting legitimacy of content related to data collection according to the embodiments of the present disclosure as described above.

[0138] It should be appreciated that all operations in the method described above are merely exemplary, and the present disclosure is not limited to any operations in the method or the order of these operations, but should cover all other equivalent changes under the same or similar concept.

[0139] It should also be appreciated that all modules in the above described device can be implemented in various ways. These modules can be implemented as hardware, software, or a combination thereof. In addition, any module in these modules can be further divided into submodules or combined together in function.

[0140] Processors have been described in conjunction with various devices and methods. These processors can be implemented using electronic hardware, computer software or any combination thereof. Whether these processors are implemented as hardware or software will depend on specific application and the overall design constraints imposed on the system. As an example, the processor provided in the present disclosure, any part of the processor or any combination of processors can be realized using a microprocessor, a microcontroller, a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic device (PLD), a state machine, a gated logic unit, a discrete hardware circuit, and other suitable processing components configured to perform the various functions described in the present disclosure. The function of the processor provided in the present disclosure, any part of the processor or any combination of processors can be realized using software executed by a microprocessor, a microcontroller, a DSP or other suitable platforms.

[0141] Software should be broadly considered to mean instructions, instruction sets, codes, code segments, program codes, programs, subroutines, software modules, applications, software applications, software packages, routines, subroutines, objects, running threads, processes, functions, etc. Software can reside in a computer-readable medium. A computer-readable medium can include, for example, a memory, which can be, for example, a magnetic storage device (e.g., a hard disk, a floppy disk, a magnetic stripe), an optical disk, a smart card, a flash memory device, a random access memory (RAM), a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), a register, or a removable disk. Although the memory is shown as being separated from the processor in the multiple aspects provided in the present disclosure, the memory can also be located inside the processor, such as a cache or a register.

[0142] The above description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Therefore, the claims are not intended to be limited to the aspects shown herein. All structural and functional equivalents to the elements of the various aspects described in this disclosure that are known or to be known to those of ordinary skill in the art are expressly incorporated herein and covered by the claims.

Claims

1. A method for detecting the legitimacy of content related to data collection, comprising: Obtain content related to data collection; detecting at least one identifying element from the content; identifying an entity corresponding to the identifying element; as well as Determining the legitimacy of the content based at least on the relevance between the creator of the content and the entity, wherein determining the legitimacy of the content comprises: Assessing that the creator of the content is associated with the entity by determining that the creator of the content is the same as the owner of the entity or determining that the creator of the content is authorized by the owner of the entity to use identifying elements of the entity; as well as Responsive to evaluating that the creator of the content is associated with the entity, the content is determined to be legitimate.

2. The method according to claim 1, wherein: The content is obtained during creation of the content.

3. The method of claim 2, further comprising, in response to determining that the content is illegal, performing at least one of the following: Sending a reminder message to the creator and / or data collection service provider; and Prevent said content from being published.

4. The method according to claim 1, wherein: The identifying element is different from the actual identifying element of the entity.

5. The method according to any one of claims 1, 2 or 4, wherein: The identification element includes a text identification element, and the text identification element is detected from the content or information related to the content.

6. The method according to any one of claims 1, 2 or 4, wherein: The identifying element comprises an image identifying element, and the detecting at least one identifying element comprises: extracting an image from the content; and The image identifying element is obtained from the image.

7. The method according to claim 6, wherein: The image extraction comprises: The image is extracted from a predetermined position of the content.

8. The method according to claim 6, wherein: The obtaining of the image identification element comprises: determining a position of the image identifying element in the image; and The image identifying element is obtained from the position.

9. The method according to claim 6, wherein: The obtaining of the image identification element comprises: Obtaining a set of sub-images by scanning the image using a sliding window; and The image identifying element is obtained from the group of sub-images.

10. The method according to any one of claims 1, 2 or 4, wherein: The identifying element comprises an image identifying element, and the identifying entity comprises: identifying at least one candidate entity corresponding to at least one visual component of the image identifying element; and The entity is determined based at least on the at least one candidate entity.

11. The method according to claim 10, further comprising: determining at least one position of the at least one visual component, and Therein, the entity is further determined based on the at least one location.

12. The method according to claim 10, wherein: The at least one visual component includes a textual visual component, and the at least one candidate entity is identified by optical character recognition (OCR).

13. The method according to claim 10, wherein: The at least one visual component includes a non-textual visual component, and the at least one candidate entity is identified by an entity recognition model.

14. The method according to claim 13, wherein: The entity recognition model is a deep learning-based model trained using a visual component sample set, each visual component sample in the visual component sample set having an entity annotation.

15. The method according to claim 14, wherein: The visual component samples in the visual component sample set are obtained by at least one of the following methods: Extracted from real identifying elements; Transforming the visual components of real iconic elements; and Created for a specific entity.

16. The method according to claim 15, wherein: The transforming of the visual component comprises at least one of the following: rotating the visual component; flipping the visual component; changing the color of the visual component to another color; and The color of the visual component is converted to grayscale.

17. The method according to any one of claims 1, 2 or 4, wherein: The entity is identified based at least on a context of the identifying element.

18. The method according to any one of claims 1, 2 or 4, further comprising: identifying reference entities corresponding to other elements in the content, the other elements comprising at least one of a template, a style, and a color; as well as determining a final entity based on the entity corresponding to the identifying element and the reference entity, and The legitimacy of the content is determined based at least on the relevance between the creator and the final entity.

19. A device for detecting the legitimacy of content related to data collection, comprising: A content acquisition module, used to obtain content related to data collection; An identification element detection module, used to detect at least one identification element from the content; An entity recognition module, used to recognize an entity corresponding to the identifying element; as well as A legitimacy determination module is used to determine the legitimacy of the content based at least on the correlation between the creator of the content and the entity, wherein determining the legitimacy of the content includes: Assessing that the creator of the content is associated with the entity by determining that the creator of the content is the same as the owner of the entity or determining that the creator of the content is authorized by the owner of the entity to use identifying elements of the entity; as well as Responsive to evaluating that the creator of the content is associated with the entity, the content is determined to be legitimate.

20. A device for detecting the legitimacy of content related to data collection, comprising: at least one processor; as well as a memory storing computer executable instructions that, when executed, cause the at least one processor to: Get content related to data collection, detecting at least one identifying element from the content, identifying the entity corresponding to the identifying element, and Determining the legitimacy of the content based at least on the relevance between the creator of the content and the entity, wherein determining the legitimacy of the content comprises: Assessing that the creator of the content is associated with the entity by determining that the creator of the content is the same as the owner of the entity or determining that the creator of the content is authorized by the owner of the entity to use identifying elements of the entity; as well as Responsive to evaluating that the creator of the content is associated with the entity, the content is determined to be legitimate.

Images