Illegal behavior presumption system, illegal behavior presumption method and storage medium

Abstract

The article information acquisition unit (105) of the illegal behavior inference system (S) of the present invention acquires article information related to an article. The identification specifying unit (106) specifies the identification of the article based on the article information. The classification specifying unit (108) specifies the classification of the article based on the article information. The inference unit (109) infers illegal behavior related to the article based on the specified identification and the specified classification.

Description

Technical Field

[0001] The present invention relates to an illegal behavior presumption system, illegal behavior presumption method and program. Background Art

[0002] In recent years, the circulation of illegal goods that use the logos of famous brands without authorization has become a problem. Patent document 1 discloses a system that attaches a tag recording information related to the goods to the goods and infers the illegal behavior of the goods by reading the information recorded on the tag.

[0003] Background Art Literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2013-214314 Summary of the invention

[0006] [Problems to be solved by the invention]

[0007] However, in the technology of Patent Document 1, since it is necessary to read a tag physically attached to an article, it is not possible to infer illegal behavior of an article posted on the Internet, for example.

[0008] The present invention has been completed in view of the above-mentioned problems, and its purpose is to provide an illegal behavior estimation system, illegal behavior estimation method and program that can estimate illegal behavior based on information related to an object without physically attaching a tag to the object or reading a tag.

[0009] [Technical means to solve the problem]

[0010] In order to solve the above-mentioned problem, the illegal behavior inference system of the present invention is characterized in that it includes: an item information acquisition mechanism for acquiring item information related to an item; an identification specific mechanism for identifying the identification of the item based on the item information; a classification specific mechanism for identifying the classification of the item based on the item information; and an inference mechanism for inferring illegal behavior related to the item based on the identified identification and the identified classification.

[0011] The illegal behavior inference method of the present invention is characterized in that it includes: an item information acquisition step, acquiring item information related to an item; an identification step, identifying the identification of the item based on the item information; a classification step, identifying the classification of the item based on the item information; and an inference step, inferring illegal behavior related to the item based on the identified identification and the identified classification.

[0012] The program of the present invention enables a computer to function as the following mechanisms: an article information acquisition mechanism that acquires article information related to an article; an identification specific mechanism that specifies the identification of the article based on the article information; a classification specific mechanism that specifies the classification of the article based on the article information; and an inference mechanism that infers illegal behavior related to the article based on the specified identification and the specified classification.

[0013] According to one aspect of the present invention, the article information includes an article image showing the article, and the identifier specifying means specifies the identifier of the article based on the article image.

[0014] According to one form of the present invention, it is characterized in that: the illegal behavior inference system also includes an identification identifier production mechanism, which produces an identification identifier based on an image showing the identification to be identified, and the identification identification mechanism specifies the identification of the object based on the object image and the identification identifier.

[0015] According to one form of the present invention, it is characterized in that: the illegal behavior presumption system also includes a retrieval mechanism, which sets the logo to be identified as a query and retrieves an image showing the logo to be identified on the Internet, and the logo identifier production mechanism produces the logo identifier based on the retrieved image.

[0016] According to one aspect of the present invention, the article information includes an article image showing the article, and the category specifying means specifies the category of the article based on the article image.

[0017] According to one form of the present invention, it is characterized in that: the illegal behavior inference system also includes a classification identifier production mechanism, which produces a classification identifier based on an image of a subject showing a classification to be identified, and the classification identification mechanism identifies the classification of the object based on the object image and the classification identifier.

[0018] According to one aspect of the present invention, the category specifying means specifies the category of the article from a plurality of predetermined categories, and the category identifier creating means creates the category identifier based on the plurality of categories.

[0019] According to one form of the present invention, it is characterized in that: the identification specifying mechanism specifies the identification of the object based on the object image, the illegal behavior inference system also includes a position information acquisition mechanism, which obtains position information related to the position of the specified identification in the object image, and the classification specifying mechanism specifies the classification of the object based on the object image and the position information.

[0020] According to one aspect of the present invention, the category specifying means processes a portion of the object image that is determined based on the position information, and specifies the category of the object based on the processed image.

[0021] According to one form of the present invention, it is characterized in that: the illegal behavior inference system also includes a feature value calculator production mechanism, which produces a feature value calculator for calculating language feature values, and the inference mechanism infers illegal behavior related to the object based on the feature value of the specific identification calculated by the feature value calculator and the feature value of the specific classification calculated by the feature value calculator.

[0022] According to one aspect of the present invention, the feature value calculator producing unit produces the feature value calculator based on a description of a genuine article.

[0023] According to one form of the present invention, it is characterized in that: the illegal behavior inference system also includes an associated data acquisition mechanism, which acquires associated data that establishes an association between each of a plurality of identifiers and at least one classification, and the inference mechanism infers illegal behavior related to the object based on the specified identifier, the specified classification and the associated data.

[0024] According to one form of the present invention, it is characterized in that: the item is a commodity, the item information is commodity information related to the commodity, the identification specific mechanism specifies the identification of the commodity based on the commodity information, the classification specific mechanism specifies the classification of the commodity based on the commodity information, and the inference mechanism infers illegal behavior related to the commodity.

[0025] [Effects of the Invention]

[0026] According to the present invention, illegal behavior can be estimated based on information related to the article without physically attaching a tag to the article or reading the tag. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a diagram showing the overall structure of the illegal behavior presumption system.

[0028] Figure 2 It is a diagram showing an article image of a genuine article.

[0029] Figure 3 A diagram showing an object image of an illegal object.

[0030] Figure 4 This is a functional block diagram showing an example of functions implemented by the illegal behavior estimation system.

[0031] Figure 5 This is a diagram showing a data storage example of an article database.

[0032] Figure 6 It is a diagram showing a data storage example of a logo image database.

[0033] Figure 7 This is a diagram showing a data storage example of a classification image database.

[0034] Figure 8 The figure shows a state where processing is performed on a logo portion in an article image.

[0035] Fig. 9 This is a flowchart showing an example of preliminary processing.

[0036] Fig.10 This is a flowchart showing an example of the estimation process.

[0037] Fig.11 This is a functional block diagram of variation (1).

[0038] Fig.12 This is a diagram showing a data storage example of related data. DETAILED DESCRIPTION

[0039] [1. Overall structure of the illegal behavior presumption system]

[0040] Hereinafter, an example of an embodiment of the illegal activity estimation system of the present invention will be described. Figure 1 is a diagram showing the overall structure of the illegal behavior presumption system. Figure 1 As shown, the illegal behavior estimation system S includes a server 10, a user terminal 20, and an administrator terminal 30, which can be connected to a network N such as the Internet. Figure 1 In the figure, one server 10, one user terminal 20, and one administrator terminal 30 are shown, but there may be a plurality of them.

[0041] The server 10 is a server computer. The server 10 includes a control unit 11, a storage unit 12 and a communication unit 13. The control unit 11 includes at least one processor. The control unit 11 performs processing according to the program or data stored in the storage unit 12. The storage unit 12 includes a main storage unit and an auxiliary storage unit. For example, the main storage unit is a volatile memory such as RAM (Random Access Memory), and the auxiliary storage unit is a non-volatile memory such as ROM (Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), flash memory or hard disk. The communication unit 13 is a communication interface for wired communication or wireless communication, and performs data communication via a network N.

[0042] The user terminal 20 is a computer operated by a user. For example, the user terminal 20 is a mobile phone (including a smart phone), a portable information terminal (including a tablet computer), or a personal computer. In this embodiment, the user terminal 20 includes a control unit 21, a storage unit 22, a communication unit 23, an operation unit 24, and a display unit 25. The physical structures of the control unit 21, the storage unit 22, and the communication unit 23 can be the same as those of the control unit 11, the storage unit 12, and the communication unit 13, respectively.

[0043] The operation unit 24 is an input device, such as a touch panel or a pointing device such as a mouse, a keyboard, or a button. The operation unit 24 transmits the user's operation content to the control unit 21. The display unit 25 is, for example, a liquid crystal display unit or an organic EL (Electroluminescence) display unit. The display unit 25 displays an image according to the instruction of the control unit 21.

[0044] The administrator terminal 30 is a computer operated by an administrator. For example, the administrator terminal 30 is a mobile phone (including a smart phone), a portable information terminal (including a tablet computer), or a personal computer. In the present embodiment, the administrator terminal 30 includes a control unit 31, a storage unit 32, a communication unit 33, an operation unit 34, and a display unit 35. The physical structures of the control unit 31, the storage unit 32, the communication unit 33, the operation unit 34, and the display unit 35 may be the same as those of the control unit 21, the storage unit 22, the communication unit 23, the operation unit 24, and the display unit 25, respectively.

[0045] In addition, the programs and data described as being stored in the storage units 12, 22, and 32 may also be supplied via the network N. In addition, the hardware configuration of each computer described above is not limited to the examples described above, and various hardware may be applied. For example, a reading unit (such as an optical drive or a memory card slot) for reading a computer-readable information storage medium, or an input / output unit (such as a USB (Universal Serial Bus) port) for inputting and outputting data with an external machine may also be included. For example, programs or data stored in an information storage medium may also be supplied to each computer via a reading unit or an input / output unit.

[0046] [2. Overview of the Illegal Behavior Presumption System]

[0047] In this embodiment, the processing of the illegal behavior estimation system S is described by taking the scenario in which the user operates the user terminal 20 to post on an SNS (Social Network Site) or a bulletin board as an example. When the server 10 receives a specified request from the administrator terminal 30, it analyzes the object image contained in the user's post to identify the identification and classification of the object, and estimates the illegal behavior related to the object based on their combination.

[0048] The object image is an image showing an object. In other words, the object image is an image in which the object is the subject. The object is photographed in the object image. The object image can be the captured image itself generated by the camera, or it can be an image after the captured image is processed. In this embodiment, the object image of the object photographed by the user is uploaded to the server 10.

[0049] An item is an object with a logo. In other words, an item is the subject of an item image. An item may or may not be a transaction object. An item may be any object, such as clothing, food, furniture, home appliances, stationery, toys, daily necessities, or transportation. An item may have a logo printed directly on it, or may have a sheet or cloth with a logo printed on it attached to it. In addition, an item is not limited to tangible objects, but may also be intangible objects such as images or animations.

[0050] A logo is identification information of an item. A logo is sometimes also called a logo or mark. For example, a logo includes a character string such as a product name, manufacturer name, seller name, brand name, store name, or affiliated organization name. For example, it also includes graphics representing a product, manufacturer, seller, brand, store, or affiliated organization. Logos are not limited to text and graphics, and can be, for example, marks, three-dimensional shapes, colors, or sounds, or a combination of these. Logos can be presented in a flat or three-dimensional form. In addition, a logo can have no particular change in appearance, or it can have a change in appearance. For example, a logo can be an animation whose appearance changes over time, or it can be a hologram whose appearance changes according to the angle.

[0051] Classification is information that indicates the type or nature of an item. Classification is sometimes also called type, category, label, distinction, or attribute. The classification can be determined based on the purpose of the item, for example, the item belongs to at least one of multiple predetermined classifications. An item can belong to only one classification or multiple classifications. In addition, classification can be defined in a hierarchical manner or not.

[0052] The so-called illegal acts related to articles refer to the unnatural combination of the identification and classification of articles. In other words, illegal acts related to articles mean the combination of identification and classification that is not considered in the articles provided by the legal right holder. For example, marking the identification of the right holder on the classification of articles that are not manufactured or licensed by the right holder who is entitled to use the identification is equivalent to illegal acts related to articles. In other words, marking the identification of articles with classifications different from those of regular articles is equivalent to illegal acts related to articles.

[0053] The so-called presumption of illegal behavior related to an article may mean presumpting that the combination of the identification and classification of an article is unnatural (for example, only processing is performed to determine whether the combination of the identification and classification of the article is unnatural, but not processing to output whether it is an illegal article), or it may also mean presuming whether it is an illegal article.

[0054] Users may post reviews of legitimate items, and may also purchase counterfeit (pirated) and other illegal items and post reviews. If illegal items are posted, it may be disadvantageous to the legal right holder of the mark, or provide wrong information to other users. Therefore, the server 10 analyzes the item image to infer whether it is a legitimate item or an illegal item.

[0055] Figure 2 is a diagram showing an image of a legitimate item. Here, a shoe manufacturer marks its own brand of shoes with a star mark m1 and sells them as an example. The server 10 analyzes the item image I1 posted by the user and identifies the mark m1 and the category of the item i1 (here, shoes). The identification method will be described below. Figure 2 In the example, since the shoe manufacturer's logo m1 is marked on the shoes sold by the shoe manufacturer, the combination of the logo and the item is a natural (appropriate) combination. Therefore, the server 10 infers that the item i1 is a legitimate item and not an illegal item.

[0056] Figure 3 is a diagram showing an image of an illegal item. Here, the shoe manufacturer has never sold a cup with its own brand logo m1, nor has it provided it as a novelty item. The server 10 analyzes the item image I2 posted by the user, and identifies the logo m1 and the category of the item i2 (here, the cup). Because the shoe manufacturer has never sold a cup with its own brand logo m1, nor has it provided it as a novelty item, they are an unnatural combination, and it is very likely that a malicious person has arbitrarily borrowed the logo to make a counterfeit. Therefore, the server 10 infers that the item i1 is an illegal item.

[0057] As described above, the illegal behavior inference system S of this embodiment analyzes the object image to identify the identification and classification. If their combination is natural, the illegal behavior inference system S infers that the object shown in the object image is a regular object. On the other hand, if their combination is unnatural, the illegal behavior inference system S infers that the object shown in the object image is an illegal object. As a result, the administrator's effort to visually judge the object image to infer illegal behavior can be reduced. The following is a detailed description of the illegal behavior inference system S.

[0058] [3. Functions of the illegal behavior presumption system]

[0059] Figure 4 1 is a functional block diagram showing an example of functions implemented by the illegal behavior estimation system S. Figure 4 As shown, in the server 10, a data storage unit 100, a retrieval unit 101, an identification identifier production unit 102, a classification identifier production unit 103, a feature value calculator production unit 104, an object image acquisition unit 105, an identification specific unit 106, a position information acquisition unit 107, a classification specific unit 108 and an inference unit 109 are implemented.

[0060] [3-1. Data storage unit]

[0061] The data storage unit 100 is mainly realized by the storage unit 12. The data storage unit 100 stores data necessary for executing the processing described in this embodiment. Here, as an example of data stored in the data storage unit 100, an article database DB1, a marking image database DB2, and a classification image database DB3 are described.

[0062] Figure 51 is a diagram showing an example of data storage in the article database DB1. Figure 5 As shown, the object database DB1 is a database storing information related to the object that is the object of illegal behavior estimation. For example, the object database DB1 stores an object ID (identifier) ​​that uniquely identifies the object, an uploaded object image, an object description, identification information for identifying the identification specified by the identification specifying unit 106, classification information for identifying the classification specified by the classification specifying unit 108, and the estimation result of the estimation unit 109.

[0063] The description is an article related to the item, for example, describing the characteristics or feelings of the item. In this embodiment, the user can freely input the description, and the description can also be a fixed phrase selected by the user. The identification information can be, for example, an ID that uniquely identifies the identifier, or a string that represents the identifier, as long as it is information that can identify the item identifier. Similarly, the classification information can be, for example, an ID that uniquely identifies the classification, or a string that represents the classification, as long as it is information that can identify the item classification. In addition, in addition to the description, a table or image that represents the characteristics of the item can also be stored in the item database DB1, and information that identifies the identifier or classification specified by the user for the item can also be stored in the item database DB1.

[0064] Figure 6 2 is a diagram showing an example of data storage in the logo image database DB2. Figure 6 As shown, the logo image database DB2 is a database storing logo images, for example, logo information and at least one logo image. As described below, in this embodiment, because the logo image is obtained through network search, the logo information can be a string set as a query during the search, or it can be an ID obtained by converting the string.

[0065] The identification image is an image used to create the identification identifier M1 described below. In principle, the identification image is an image of a legitimate article, but it may also contain some images of illegal articles. The identification image may be an image of an article stored in the article database DB1, or an image not stored in the article database DB1. In this embodiment, the identification image retrieved by the retrieval unit 101 described below is stored in the identification image database DB2. When the identification identifier M1 described below is learning, the learning may be performed after color correction or image restoration or other processing is performed on the parts other than the identification part in the identification image, or the learning may be performed without performing such processing.

[0066] Figure 7 1 is a diagram showing an example of data storage in the classified image database DB3. Figure 7As shown, the classified image database DB3 is a database storing classified images. For example, at least one classified image is stored in the classified image database DB3 for each classification information.

[0067] The classification image is an image used to create the classification identifier M2 described below. The classification image is an image used to make the classification identifier M2 learn the shape of a general object. In the present embodiment, only the object is shown in the classification image without showing the logo, but in the classification image, the object with the logo can also be shown in the classification image. In this case, when the classification identifier M2 described below performs learning, the learning can be performed after color correction or image restoration or the like is performed on the logo part in the classification image, or the learning can be performed without performing such processing.

[0068] In addition, the classification image may be an image of an item stored in the item database DB1 or an image not stored in the item database DB1. In the present embodiment, the case where an image downloaded from another system that provides images for research is equivalent to the classification image is described, but it may not be an image downloaded from another system, and the classification image may be prepared by the administrator himself.

[0069] In addition, the data stored in the data storage unit 100 is not limited to the above example. For example, the data storage unit 100 stores a marker identifier M1 for identifying a marker. The marker identifier M1 includes a program (algorithm) or parameters, etc. In the present embodiment, the machine learning model used in image recognition is used as an example for explanation. Machine learning itself can apply various well-known methods, such as CNN (Convolutional Neural Network), ResNet (Residual Network), or RNN (Recurrent Neural Network). When an object image or its feature value is input, the marker identifier M1 outputs the marker in the object image and the position information related to its position. In addition, the marker identifier M1 may not output the position information of the marker.

[0070] For example, in addition to the examples described above, the marker identifier M1 can also use methods called CAM (Class Activation Mapping), YOLO (You Only Look Once) or SSD (SingeShot MultiBox Detector). According to these methods, both the identification result of the marker and the information related to the part focused on during identification (such as a heat map) can be output. For example, when the marker position (such as a bounding box) in the image is annotated, if YOLO or SSD is used, not only the marker can be detected, but also its position can be detected. In addition, for example, even when the marker position is not annotated, by using a method called Grad-CAM (Gradient-weighted Class Activation Mapping) separately from the marker identifier M1, information related to the part focused on by the marker identifier M1 for marker identification (such as a heat map) can be output, so the approximate position of the marker can be inferred.

[0071] In addition, for example, the data storage unit stores the classification identifier M2. The classification identifier M2 includes a program (algorithm) or parameters, etc. In the present embodiment, the machine learning model used in image recognition is used as an example for explanation. The classification identifier M2 is the same as the identification identifier M1, and is also a machine learning method that can apply various known methods. For example, the classification identifier M2 can also use methods such as CNN, ResNet or RNN. When an object image or its feature value is input, the classification identifier M2 outputs the classification of the object shown in the object image.

[0072] In addition, for example, the data storage unit stores the feature value calculator M3. The feature value calculator M3 includes a program (algorithm), parameters, and dictionary data for converting vocabulary into feature values. In the present embodiment, the machine learning model used in natural language processing is used as an example for explanation. The feature value calculator M3 is the same as the identification identifier M1 and the classification identifier M2, and various well-known methods can also be applied. For example, the feature value calculator M3 can use a method called Word2Vec or Glove. When a character string is input, the feature value calculator M3 outputs a feature value representing its meaning. In the present embodiment, the feature value is represented in the form of a vector, but the feature value can be represented in any form, for example, in the form of a sequence, or by a single numerical value.

[0073] [3-2. Search section]

[0074] The search unit 101 is mainly implemented by the control unit 11. The search unit 101 sets the mark to be identified as a query and searches for images showing the mark to be identified on the Internet. The search itself can use various well-known search engines provided by portal websites, etc. In addition, the search range can be any range, for example, it can be a range that can be searched from a portal website (the entire range on the Internet), or it can be a range of a specific database such as an online shopping mall.

[0075] For example, the search unit 101 obtains an identification character string input by an administrator from the administrator terminal 30, sets the obtained character string as a query, and performs image search. The search unit 101 associates all images or part of the images found in the search with the trademark that becomes the query, and stores them as identification images in the identification image database DB2. For example, the search unit 101 obtains a specified number of images in descending order of scores during the search, and stores them as identification images in the identification image database DB2. In addition, for example, an image randomly selected from the search results is stored as an identification image in the identification image database DB2. In addition, for example, the search unit 101 causes the display unit 35 of the administrator terminal 30 to display the search results, and stores the image selected by the administrator as an identification image in the identification image database DB2.

[0076] In addition, the search unit 101 only needs to store at least one identification image in the identification image database DB2, and any number of identification images is acceptable. For example, the search unit 101 may store a predetermined number of identification images in the identification image database DB2, or may store all identification images or part of identification images that are scored above a threshold value during retrieval in the identification image database DB2. In addition, in the present embodiment, the case where the identification string is set as a query is described, but an image showing the identification may also be set as a query to retrieve similar images. In this case, there may be only one image that serves as a query, or a plurality of images that have mutually changed the conditions or angles of illumination of the identification may be set as queries.

[0077] [3-3. Logo recognition device production department]

[0078] The marker identifier creation unit 102 is mainly implemented by the control unit 11. The marker identifier creation unit 102 creates the marker identifier M1 based on the marker image showing the marker to be recognized. The creation of the marker identifier M1 refers to adjusting the mode of the marker identifier M1, for example, adjusting the algorithm or parameters of the marker identifier M1. In this embodiment, since the marker image is retrieved by the search unit 101, the marker identifier creation unit 102 creates the marker identifier M1 based on the retrieved image.

[0079] For example, based on the identification image stored in the identification image database DB2, the identification identifier production unit 102 obtains teaching data that uses the identification image or its feature value as input and the identification shown in the identification image as output. The identification identifier production unit 102 causes the identification identifier M1 to learn based on the obtained teaching data. The learning itself can use the method used in well-known machine learning, for example, it can use the learning method of CNN, ResNet or RNN. The identification identifier production unit 102 produces the identification identifier M1 in a manner that obtains the relationship between the input and output shown in the teaching data.

[0080] [3-4. Classification Identifier Production Department]

[0081] The classification identifier creating unit 103 is mainly implemented by the control unit 11. The classification identifier creating unit 103 creates the classification identifier M2 based on the image showing the subject of the classification to be identified. The creation of the classification identifier M2 refers to adjusting the mode of the classification identifier M2, for example, adjusting the algorithm or parameters of the classification identifier M2. In this embodiment, since a classification image obtained from another system is prepared, the identification identifier creating unit 102 creates the classification identifier M2 based on the classification image.

[0082] For example, the classification identifier production unit 103 obtains teaching data that uses the classification image or its feature value as input and the classification shown in the classification image as output based on the classification image stored in the classification image database DB3. The identification identifier production unit 102 causes the identification identifier M1 to learn based on the obtained teaching data. The learning itself can use the method used in well-known machine learning, for example, it can use the learning method of CNN, ResNet or RNN. The classification identifier production unit 103 produces the classification identifier M2 in a manner that obtains the relationship between the input and output shown in the teaching data.

[0083] In this embodiment, since a plurality of categories are prepared in advance, the category identifier creation unit 103 creates the category identifier M2 based on the category information of each of the plurality of categories. The category can be any category specified by the administrator, for example, the type or category of goods listed in the online shopping mall. The category identifier creation unit 103 adjusts the category identifier M2 in a manner to output the category information of any one of the plurality of predetermined categories.

[0084] [3-5. Eigenvalue calculator production department]

[0085] The feature value calculator creating unit 104 is mainly implemented by the control unit 11. The feature value calculator creating unit 104 creates a feature value calculator M3 for calculating language feature values. Creating the feature value calculator M3 refers to adjusting the mode of the feature value calculator M3, for example, adjusting the algorithm or parameters of the feature value calculator M3, or creating dictionary data of the feature value calculator M3.

[0086] The method for creating the feature value calculator M3 itself can use a well-known method, for example, a method called Word2Vec or Glove. For example, the feature value calculator creating unit 104 can create the feature value calculator M3 based on the description of a legitimate article. A legitimate article is an article that is determined to be legitimate by the estimation unit 109. For example, the feature value calculator creating unit 104 creates the feature value calculator M3 based on the description stored in the article database DB1.

[0087] In addition, the feature value calculator creating unit 104 may acquire a document database from other systems to create the feature value calculator M3, or may acquire a document database prepared by an administrator to create the feature value calculator M3, instead of acquiring the description text stored in the item database DB1. The document database may utilize any database, such as articles on a website providing an encyclopedia, articles on an integrated website, or product description texts in an online shopping mall.

[0088] [3-6. Article image acquisition unit]

[0089] The article image acquisition unit 105 is mainly implemented by the control unit 11. The article image acquisition unit 105 acquires an article image showing an article. For example, the article image acquisition unit 105 refers to the article database DB1 and acquires the article image to be processed. The article image acquisition unit 105 may acquire only one article image or multiple article images, as long as at least one article image is acquired.

[0090] In addition, the item image is an example of information included in the item information. Therefore, the item image in this embodiment may be referred to as item information. The item information only needs to include information related to the item, and in addition to the image, it may include other information such as a character string, a table, a diagram, an animation, or a sound, and may include multiple of these information.

[0091] [3-7. Identification section]

[0092] The identification specific unit 106 is mainly implemented by the control unit 11. The identification specific unit 106 identifies the identification of the object based on the object image. The identification here refers to extracting the identification of the object from the object image. The identification specific unit 106 can identify the identification in the form of a string or ID, or in the form of an image.

[0093] In this embodiment, since the identifier M1 is produced by the identifier producing unit 102, the identifier specifying unit 106 specifies the identifier of the item based on the item image and the identifier M1. The identifier specifying unit 106 inputs the item image or its feature value into the identifier M1. The identifier M1 outputs the identifier information for identifying the identifier shown in the item image based on the input item image or feature value. The identifier specifying unit 106 specifies the identifier of the item by acquiring the output of the identifier M1.

[0094] In addition, the identification method is not limited to the method using the identification identifier M1, and various image analysis technologies can be used. For example, the identification identification unit 106 can also use pattern matching with the sample image to identify the identification of the item based on the item image. In this case, the sample image showing the basic shape of the identification is stored in the data storage unit 100, and the identification identification unit 106 identifies the identification of the item by determining whether there is a portion similar to the sample image in the item image. In addition, for example, the identification identification unit 106 can also extract feature points or contour lines from the item image and identify the identification of the item based on the pattern of the feature points or contour lines.

[0095] [3-8. Location information acquisition unit]

[0096] The position information acquisition unit 107 is mainly implemented by the control unit 11. The position information acquisition unit 107 acquires position information related to the position of the marker identified in the object image. In this embodiment, since the marker identifier M1 outputs position information related to the position of the marker shown in the object image, the position information acquisition unit 107 acquires the position information output by the marker identifier M1.

[0097] The position information is information indicating the position of the image portion showing the mark in the article image. In the present embodiment, the case where the position information indicates the position of the bounding box surrounding the mark is described, but the position information may also indicate the position of any pixel showing the mark instead of the position of the bounding box. For example, the position information is represented by coordinate information of a two-dimensional coordinate axis set for the article image. The two-dimensional coordinate axis may be set with a predetermined position of the article image as the origin, for example, with the upper left corner of the article image as the origin, the X axis set in the right direction, and the Y axis set in the downward direction.

[0098] In addition, when the logo is identified by pattern matching instead of by the logo identifier M1, the position information acquisition unit 107 may also acquire the position information by identifying a portion of the object image that is similar to the sample image. In addition, for example, the position information acquisition unit 107 may also acquire the position information by identifying a feature point or contour line in the object image that is estimated to be a logo portion.

[0099] [3-9. Classification Specific Part]

[0100] The category specifying unit 108 is mainly implemented by the control unit 11. The category specifying unit 108 specifies the category of the object based on the object image. The specification here means determining the category to which the object shown in the object image belongs among multiple categories. The category specifying unit 108 specifies the category of the object from multiple predetermined categories.

[0101] In this embodiment, since the classification identifier M2 is created by the classification identifier creation unit 103, the classification identification unit 108 identifies the classification of the object based on the object image and the classification identifier M2. The classification identification unit 108 inputs the object image or its feature value to the classification identifier M2. The classification identifier M2 outputs classification information for identifying the classification of the object shown in the object image based on the input object image or feature value. The classification identification unit 108 identifies the classification of the object by acquiring the output of the classification identifier M2.

[0102] In addition, in the present embodiment, the classification specifying unit 108 specifies the classification of the item based on the item image and the position information. For example, the classification specifying unit 108 processes the portion of the item image determined based on the position information, and specifies the classification of the item based on the image after the processing. The processing here can be any image processing that reduces or eliminates the features of the identification part, such as color correction of the identification part; filling the identification part with a specified color or the surrounding color; or blurring the identification part. In addition, for example, not only color but also texture or shape can be used to fill in a way that is harmonious with the surrounding (so-called filling corresponding to the content).

[0103] Figure 8 is a diagram showing the processing of the marked part in the object image. Figure 8 As shown, for example, the category specifying unit 108 specifies the category of the object after color correction or other processing is performed on the boundary box b1 representing the identification part in the object image 12. The category specifying unit 108 specifies the category of the object by inputting the processed object image or its feature value into the category identifier M2 and obtaining the output of the category identifier M2.

[0104] In addition, the method of specifying the classification is not limited to the method using the classification identifier M2, and various image analysis technologies can be used. For example, the classification specifying unit 108 can also use pattern matching with the sample image to specify the classification of the object based on the object image. In this case, the sample image representing the basic shape of the object belonging to each classification is stored in the data storage unit 100, and the classification specifying unit 108 specifies the classification of the object by determining whether there is a portion similar to the sample image from the object image. In addition, for example, the classification specifying unit 108 can also extract feature points or contour lines from the object image and specify the classification of the object based on the pattern of the feature points or contour lines.

[0105] [3-10. Estimation section]

[0106] The inference unit 109 is mainly implemented by the control unit 11. The inference unit 109 infers illegal behavior related to the article based on the identification specified by the identification specifying unit 106 and the classification specified by the classification specifying unit 108. For example, the inference unit 109 determines whether the combination of the identification and the classification is a natural (appropriate) combination. The inference unit 109 infers that there is no illegal behavior related to the article when the combination of the identification and the classification is a natural combination, and infers that there is illegal behavior related to the article when the combination of the identification and the classification is an unnatural combination.

[0107] In addition, for example, the inference unit 109 determines whether the combination of the identification and classification of the item meets the prescribed criterion. The criterion can be any criterion for determining whether the item is illegal. In the present embodiment, it is described as a case where it is a criterion for the distance between the identification feature value and the classification feature value. In addition, the criterion is not limited to the distance between the feature values, and it is also possible to determine whether the identification and classification are a prescribed combination as in the following variation. In addition, for example, a machine learning model can be prepared that sets the identification and classification as input and the inference result of illegal behavior as output, and the inference unit 109 infers illegal behavior by using the machine learning model.

[0108] In the present embodiment, the inference unit 109 infers illegal behavior related to the article based on the feature value of the identifier calculated by the feature value calculator M3 and the feature value of the classification calculated by the feature value calculator M3. For example, the inference unit 109 inputs the character string representing the identifier specified by the identifier specifying unit 106 into the feature value calculator M3, and obtains the feature value calculated by the feature value calculator M3. In addition, for example, the inference unit 109 inputs the character string representing the classification specified by the classification specifying unit 108 into the feature value calculator M3, and obtains the feature value calculated by the feature value calculator M3. In addition, in the case where the identification information is a character string, the character string is directly input into the feature value calculator M3, and in the case where the identification information is an ID, the ID is converted into a character string and then input into the feature value calculator M3. Similarly, in the case where the classification information is a character string, the character string is directly input into the feature value calculator M3, and in the case where the classification information is an ID, the ID is converted into a character string and then input into the feature value calculator M3.

[0109] The estimation unit 109 determines whether the difference between the feature value of the identification and the feature value of the classification is greater than a threshold value. In the present embodiment, since the feature value is represented in a vector form, the difference is a distance in the vector space. When the feature value is represented in another form, the difference may be a difference in numerical values. When the difference is less than the threshold value, the estimation unit 109 estimates that there is no illegal behavior related to the article, and when the difference is greater than the threshold value, it estimates that there is illegal behavior related to the article. The estimation unit 109 associates the estimation result with the article image and stores it in the article database DB1.

[0110] Furthermore, when an item is estimated as illegal by the estimation unit 109, any processing may be performed afterwards. For example, the administrator terminal 30 may display a list of item images estimated as illegal, and the item images selected by the administrator may be deleted from the server 10. In addition, for example, the administrator may contact the user who posted the item image estimated as illegal by e-mail or the like to confirm the item. In addition, for example, the item image estimated as illegal may be forcibly deleted from the server 10.

[0111] [4. Processing performed in this embodiment]

[0112] Next, the processing executed in this embodiment will be described. Here, the preliminary processing for creating the mark identifier M1, the classification identifier M2, and the feature value calculator M3, and the estimation processing for estimating the fraudulent behavior of the article will be described.

[0113] [4-1. Preliminary processing]

[0114] Fig. 9 This is a flowchart showing an example of preliminary processing. Fig. 9The above-mentioned preliminary processing is performed by the control unit 11, 31 according to the program stored in the storage unit 12, 32. Figure 4 An example of the processing executed by the functional modules shown. In addition, here, the case where the label identifier M1, the classification identifier M2, and the feature value calculator M3 are each created in a series of processes is described, but they can also be created in different processes.

[0115] like Fig. 9 As shown, in the administrator terminal 30, the control unit 31 sends a search request for a logo image using the logo input by the administrator as a query to the server 10 (S100). In S100, the administrator inputs a character string used as a query logo from the operation unit 34. The control unit 31 sends a search request using the character string input by the administrator as a query.

[0116] In the server 10, when receiving a search request, the control unit 11 sets the logo input by the administrator as a query and searches for logo images on the Internet (S101). Here, the control unit 11 obtains a specified number of logo images found in the search, but the search results can also be sent to the administrator terminal 30 to accept the administrator's selection.

[0117] The control unit 11 stores the logo image retrieved in S101 in the logo image database DB2 (S102). In S102, the control unit 11 associates the logo input by the administrator with the logo image acquired in S101 and stores them in the logo image database DB2.

[0118] The control unit 11 creates a marker identifier M1 based on the marker image stored in the marker image database DB2 (S103). In S103, the control unit 11 creates teaching data that takes the marker image or its feature value as input and the marker input by the administrator as output. The control unit 11 causes the marker identifier M1 to learn based on the created teaching data.

[0119] In the administrator terminal 30, the control unit 31 sends a request for creating the classification identifier M2 to the server 10 (S104). The request for creating the classification identifier M2 can be made by sending information in a predetermined format. In addition, the case where the classification image is pre-stored in the classification image database DB3 is described here, but the classification image may be included in the request for creating the classification identifier M2. In addition, for example, when the server 10 receives the request for creating the classification identifier M2, the classification image may be downloaded from another system.

[0120] In the server 10, when receiving a request to create the classification identifier M2, the control unit 11 creates the classification identifier M2 based on the classification image stored in the classification image database DB3 (S105). In S105, the control unit 11 creates teaching data that takes the classification image or its feature value as input and the classification associated with the classification image as output. The control unit 11 causes the classification identifier M2 to learn based on the created teaching data.

[0121] In the administrator terminal 30, the control unit 31 sends a request for making the feature value calculator M3 to the server 10 (S106). The request for making the feature value calculator M3 can be made by sending information in a predetermined format. In addition, the case of using the description text of the item database DB1 is described here, but the request for making the feature value calculator M3 may also include document data required for making the feature value calculator M3. In addition, for example, when the server 10 receives the request for making the feature value calculator M3, the document data may be downloaded from other systems.

[0122] In the server 10, when receiving a request to create a feature value calculator M3, the control unit 11 creates a feature value calculator M3 based on the item database DB1 (S107), and the process ends. In S107, the control unit 11 divides the description text stored in the item database DB1 into words, and uses a function for calculating feature values ​​to feature each word, thereby creating a feature value calculator M3.

[0123] [4-2. Estimation Process]

[0124] Fig.10 This is a flowchart showing an example of the estimation process. Fig.10 The estimation process shown is executed by the control unit 11, 31 operating according to the program stored in the storage unit 12, 32. The process described below is executed by Figure 4 An example of the processing performed by the functional modules shown.

[0125] like Fig.10 As shown, first, in the administrator terminal 30, the control unit 31 sends a request for executing the estimation process to the server 10 (S200). The request for executing the estimation process can be made by sending information in a predetermined format. In addition, here, the case where the object image for which the estimation result is not stored in the object database DB1 becomes the processing object is described, but the request for executing the estimation process can also include the object ID of the object image as the processing object. In addition, the estimation process can be executed at any other time point not according to the instruction of the administrator. For example, the estimation process can be executed periodically, or it can be executed in response to the accumulation of a predetermined number of object images.

[0126] In the server 10, when receiving a request to execute the estimation process, the control unit 11 acquires the item image to be processed based on the item database DB1 (S201). In S201, the control unit 11 refers to the item database DB1 and acquires any item image for which no estimation result is stored.

[0127] The control unit 11 identifies the identification and position information of the object based on the object image to be processed and the identification identifier M1 (S202). In S202, the control unit 11 inputs the object image or its feature value into the identification identifier M1. The identification identifier M1 outputs the learned identification information representing at least one of the multiple identifications and the position information of the identification based on the input object image or feature value. The control unit 11 obtains the output result of the identification identifier M1. In addition, when the identification identifier M1 does not have the function of outputting the identification position information, the control unit 11 only needs to obtain the position information using Grad-CAM or the like.

[0128] The control unit 11 identifies the classification of the object based on the object image to be processed, the identification position information obtained in S202, and the classification identifier M2 (S203). In S203, the control unit 11 performs color correction or image restoration on the area indicated by the position information in the object image. The control unit 11 inputs the processed object image or its feature value to the identification identifier M1. The identification identifier M1 outputs the learned classification information representing at least one of the multiple classifications based on the input object image or feature value. The control unit 11 obtains the classification information output by the identification identifier M1.

[0129] The control unit 11 calculates the distance between the feature value of the identification specified in S202 and the feature value of the classification specified in S203 (S204) based on the feature value calculator M3. In S204, the control unit 11 inputs the identification information into the feature value calculator M3 to obtain the feature value of the identification. The control unit 11 inputs the classification information into the feature value calculator M3 to obtain the feature value of the classification. The control unit 11 calculates the distance between the feature value of the identification and the feature value of the classification.

[0130] The control unit 11 determines whether the distance between the feature value of the identification and the feature value of the classification is greater than a threshold value (S205). The threshold value may be a fixed value or a variable value, as long as it is a predetermined value. When the threshold value is set as a variable value, it can be determined based on at least one of the identification and the classification.

[0131] When the distance is determined to be greater than the threshold value (S205; Y (Yes)), the control unit 11 infers that the item is illegal (S206). In S206, the control unit 11 associates the identification information, classification information and inference result of the item with the item image to be processed and stores them in the item database DB1.

[0132] On the other hand, when it is determined that the distance is less than the threshold value (S205; N (No)), the control unit 11 infers that the item is legitimate (S207). In S207, the control unit 11 associates the identification information, classification information and inference result of the item with the item image to be processed and stores them in the item database DB1.

[0133] The control unit 11 determines whether all the article images to be processed have been estimated based on the article database DB1 (S208). In S208, the control unit 11 determines whether there is an article image for which an estimation result has not yet been obtained.

[0134] If there is an article image that has not been estimated (S208; N), the process returns to S201 to execute the process for the next article image. On the other hand, if all article images have been estimated (S208; Y), the process ends.

[0135] According to the illegal behavior inference system S described above, the identification of the object and the classification of the object are specified based on the object image, and the illegal behavior related to the object is inferred based on their combination. Therefore, the illegal behavior can be inferred based on the information related to the object without physically installing a label on the object or reading the label. For example, in the method of installing a physical label on the object as in the prior art, the administrator must go to the store to read the label, but according to the illegal behavior inference system S, as long as there is an object image, the illegal behavior can be inferred, so the illegal behavior can be detected quickly. In other words, the process from the publication of the object image to the detection of the illegal behavior can be accelerated. In addition, the administrator does not need to visually judge the illegal behavior, which can reduce the time to infer the illegal behavior of the object. For example, even if there is no physical object of the object, the illegal behavior can be inferred based on the information such as the object image, so the administrator can save the time to personally recognize the object image to judge the illegal behavior.

[0136] In addition, even if one wants to identify the item based on information that can be freely input by the poster, such as the item description, it is difficult to detect illegal behavior because such information is easy to be falsified and untrue. From this perspective, the illegal behavior inference system S can improve the accuracy of inferring illegal behavior by identifying the item's logo based on the item image, which is relatively difficult to falsify. In addition, even if there is no information such as the description, illegal behavior can be inferred as long as there is an item image.

[0137] In addition, the fraud estimation system S creates a mark identifier M1 based on a mark image showing a mark to be identified, and identifies the mark of the object based on the object image and the mark identifier M1, thereby improving the accuracy of identifying the mark. As a result, the accuracy of estimating the fraudulent behavior of the object can also be improved.

[0138] In addition, the illegal behavior inference system S sets the logo to be identified as a query, searches for a logo image showing the logo to be identified on the Internet, and creates a logo identifier M1 based on the retrieved image, thereby making it easier to collect logo images and reducing the effort to create the logo identifier M1. In addition, by using various logo images on the Internet, the accuracy of the logo identifier M1 can be effectively improved. As a result, the accuracy of inferring illegal behavior of an item can also be improved.

[0139] In addition, even if one wants to identify the category of an item based on information that can be freely input by the poster, such as an item description, it is difficult to detect illegal behavior because such information is easy to be falsified and untrue. From this perspective, the illegal behavior inference system S can improve the accuracy of inferring illegal behavior by identifying the category of an item based on an item image that is relatively difficult to falsify. In addition, even if there is no information such as a description, illegal behavior can be inferred as long as there is an item image.

[0140] In addition, the illegal behavior estimation system S creates a classification identifier M2 based on a classification image of a subject showing a classification to be identified, and identifies the classification of the object based on the object image and the classification identifier M2, thereby improving the accuracy of identifying the classification. As a result, the accuracy of estimating illegal behavior of the object can also be improved.

[0141] In addition, the fraud estimation system S can effectively improve the accuracy of the classification identifier M2 by specifying the classification of the object from a plurality of predetermined classifications and creating the classification identifier M2 based on the plurality of classifications. As a result, the accuracy of estimating the fraudulent behavior of the object can also be improved.

[0142] In addition, the illegal behavior inference system S can effectively improve the accuracy of specific classification by acquiring position information related to the identification position in the object image and identifying the category of the object based on the object image and the position information.

[0143] In addition, the fraud estimation system S can prevent classification errors caused by strong influence of the marked part by processing the part of the object image determined based on the position information and then estimating the classification. As a result, the accuracy of estimating the fraudulent behavior of the object can be improved.

[0144] In addition, the illegal behavior estimation system S produces a feature value calculator M3 for calculating language feature values, and estimates illegal behavior related to the article based on the feature value of the specific identification calculated by the feature value calculator M3 and the feature value of the specific classification calculated by the feature value calculator M3, thereby improving the accuracy of estimating illegal behavior of the article. For example, it is also possible to prepare the association between the identification and the classification in advance as in the following variation, but in this case, when the administrator specifies an incorrect association or the association is omitted, the estimation accuracy of illegal behavior may decrease. From this perspective, by using the language feature value as an objective indicator to estimate illegal behavior of the article, it is possible to prevent the estimation accuracy of illegal behavior from decreasing.

[0145] In addition, the illegal behavior inference system S can improve the accuracy of the feature value calculator M3 by creating the feature value calculator M3 based on the description of the legitimate item. For example, a malicious user may deliberately input a false text into the description of the illegal item, but by excluding such description, the accuracy of the feature value calculator M3 can be improved. As a result, the accuracy of the inference of the illegal behavior of the item can also be improved.

[0146] [5. Variations]

[0147] In addition, the present invention is not limited to the above-described embodiment, and can be appropriately modified within the scope of the present invention.

[0148] (1) For example, the illegal behavior estimation method by the estimation unit 109 is not limited to the example described in the embodiment. The estimation unit 109 only needs to determine whether the combination of the identification and classification of the article is natural, and for example, a natural combination or an unnatural combination thereof may be prepared in advance.

[0149] Fig.11 is the functional block diagram in variation (1). Fig.11 As shown, in variation (1), the data storage unit 100 stores the associated data DT1, and in addition to realizing the functions described in the embodiment, can also realize the data acquisition unit 110. The data acquisition unit 110 is mainly realized by the control unit 11. In addition, in variation (1), the data storage unit 100 may not store the feature value calculator M3, and the illegal behavior estimation system S may not include the feature value calculator production unit 104.

[0150] Fig.12 DT1 is a diagram showing a data storage example of the associated data DT1. Fig.12As shown, the associated data DT1 stores identification information of each of the plurality of identifications and classification information of at least one classification. In other words, at least one classification information is stored for each identification information in the associated data DT1. Here, the case where a natural combination of identification and classification is defined in the associated data DT1 is described, but an unnatural combination of identification and classification may also be defined in the associated data DT1.

[0151] Here, the case where the administrator prepares the related data DT1 is described, but the related data DT1 can also be automatically generated by obtaining statistics of the item database DB1. For example, the administrator uses the catalog or homepage of the item manufacturer to specify the natural combination of the identification and the classification. The administrator inputs these combinations from the operation unit 34 of the administrator terminal 30 to create the related data DT1, and uploads it to the server 10. When receiving the related data DT1 uploaded by the administrator, the server 10 stores it in the data storage unit 100.

[0152] The data acquisition unit 110 acquires the association data DT1 that associates each of the plurality of identifiers with at least one classification. In this variation, since the association data DT1 is stored in the data storage unit 100 , the data acquisition unit 110 acquires the association data DT1 stored in the data storage unit 100 .

[0153] The inference unit 109 infers illegal behavior related to the article based on the specified identifier, the specified classification, and the associated data DT1. For example, when a natural combination of the identifier and the classification is defined in the associated data DT1, the inference unit 109 determines whether the combination of the identifier and the classification of the article exists in the associated data DT1. When their combination exists in the associated data DT1, the inference unit 109 infers that there is no illegal behavior related to the article, and when their combination does not exist in the associated data DT1, the inference unit 109 infers that there is illegal behavior related to the article.

[0154] In addition, in the case where an unnatural combination of identification and classification is defined in the associated data DT1, when the combination of the identification and classification of the object does not exist in the associated data DT1, the inference unit 109 infers that there is no illegal behavior related to the object, and when their combination exists in the associated data DT1, the inference unit 109 infers that there is illegal behavior related to the object.

[0155] According to variation (1), illegal behavior related to the article can be estimated based on the identified identifier, the identified classification, and the associated data DT1, thereby reducing the effort for estimating illegal behavior. For example, in the method described in the embodiment, the server 10 must create a feature value calculator M3 or calculate feature values, but in the method of variation (1), such processing is not required, so illegal behavior can be estimated through simple processing, and the processing load of the server 10 can also be reduced.

[0156] (2) In addition, for example, in the embodiment, the case where illegal behavior related to an item is estimated based on an image of the item posted on an SNS or a bulletin board is described, but the illegal behavior estimation system S can be applied to any other scenario. For example, the illegal behavior estimation system S can also be used in the scenario of determining illegal behavior related to goods displayed on an online shopping mall.

[0157] In this variation, the server 10 manages the website of an online shopping mall. The user who operates the user terminal 20 is a store staff or the like who has opened a branch store on the online shopping mall. The user uploads the product information related to the products displayed in the store to the server 10. The product database DB1 stores the product ID that uniquely identifies the products sold in the store, the product image, the product description, the identification information for identifying the identification specified by the identification identifier M1, the classification information for identifying the classification specified by the classification identifier M2, and the estimation result of the estimation unit 109. Based on this information, the product details page for purchasing the product is displayed.

[0158] In this variation, the article is a commodity, and the article information is commodity information related to the commodity. The commodity can be any commodity that is subject to transaction on the online shopping mall. The commodity information can be any basic information related to the commodity, and can be any information input by a user such as a store staff.

[0159] The identification specifying unit 106 specifies the identification of the commodity based on the commodity information, and the classification specifying unit 108 specifies the classification of the commodity based on the commodity information. The identification and classification specifying methods themselves are as described in the embodiment. The inference unit 109 infers illegal behavior related to the commodity. The inference method of illegal behavior can use the method described in the embodiment or the method described in variation (1). For example, the administrator does not display the purchase page of the item that is inferred to be illegal, or punishes the store that sells the item.

[0160] In addition, the item database DB1 may also store classification information for identifying the commodity classification specified by the store, commodity titles, commodity prices, commodity inventory, and other information. In this case, the classification specifying unit 108 may specify the classification of the commodity by referring to the classification information specified by the store without using the classification identifier M2. Similarly, the identification specifying unit 106 may specify the identification based on the description or title of the commodity.

[0161] According to variation (2), by identifying the identification and classification of the commodity based on the commodity information and inferring illegal behavior related to the commodity, it is possible to prevent illegal commodities from being sold.

[0162] (3) In addition, for example, in the embodiment, an item image is described as an example of item information, but the item information may also be other information such as a character string, animation, or sound. For example, in the case where the item information is a character string, the identifier specifying unit 106 may specify the identifier by determining whether the character string associated with the item contains a character string representing the identifier. In this case, the position information indicates the position of the character string of the identifier in the entire article. The classification specifying unit 108 specifies the classification by determining whether the character string associated with the item contains a character string representing the classification, or by referring to the classification information associated with the item. In addition, for example, the classification specifying unit 108 may specify the classification of the item after hiding the character string of the identifier portion indicated by the position information.

[0163] In addition, for example, when the item information is an animation, the identification unit 106 and the classification unit 108 can identify the identification and classification for each image constituting the animation by using the method described in the embodiment or the variation. In addition, for example, when the item information is a sound, the identification becomes the sound used in CM (Commercial Message, TV commercial) and the like. The identification unit 106 identifies the identification by analyzing the sound of the item information and determining whether a waveform representing the identification is obtained. The classification unit 108 can identify the classification by analyzing the sound, and when there is other information such as a character string or an image, the classification can also be identified by referring to the other information.

[0164] In addition, for example, the identifier and the category may be identified by referring to individual items included in the article information, such as the identifier identifying unit 106 identifying the identifier by referring to the article image, and the category identifying unit 108 identifying the category by referring to the description or category information.

[0165] In addition, for example, the case where the main functions are implemented by the server 10 has been described, but each function may be shared by multiple computers. For example, the server 10, the user terminal 20, and the administrator terminal 30 may share the functions. For example, processing such as classification may be performed by the user terminal 20 or the administrator terminal 30 instead of the server 10. In addition, for example, in the case where the illegal behavior estimation system S includes multiple server computers, the functions may be shared by multiple server computers. In addition, for example, data described as being stored in the data storage unit 100 may be stored in a computer other than the server 10.

Claims

1. A system for presumpting illegal behavior, characterized in that Include: The identification identifier production means produces the identification identifier by acquiring teaching data that uses the image or its feature value as input and the identification shown in the image as output based on an image showing the identification to be recognized, and causing the identification identifier to learn in machine learning based on the acquired teaching data; An item information acquisition unit acquires item information including an item image showing the item; An identification specific mechanism that identifies the identification of the object based on the object image and the identification identifier; A classification specifying mechanism specifies a classification as a type or category of the item based on the item information; as well as The inference unit infers illegal behavior related to the article based on the specified identification and the specified classification.

2. The illegal behavior presumption system according to claim 1, characterized in that: The illegal behavior presumption system further includes a search unit that sets the logo to be identified as a query and searches the Internet for a logo image showing the logo to be identified. The marker identifier creating unit creates the marker identifier based on the retrieved marker image.

3. The illegal behavior presumption system according to claim 2, characterized in that: The search unit searches for the logo image based on a database of an online shopping mall that displays the article corresponding to the article information as a commodity.

4. A system for inferring illegal behavior, characterized in that Include: A classification identifier making unit, which makes the classification identifier by learning the classification identifier in machine learning based on the acquired teaching data by acquiring teaching data that takes the image or its feature value as input and the classification shown by the image as output based on an image of a subject showing a classification to be identified; An item information acquisition unit acquires item information including an item image showing the item; An identification specific mechanism is used to identify the item based on the item information; A classification specifying mechanism, based on the object image and the classification identifier, specifies the classification of the object; as well as The inference unit infers illegal behavior related to the article based on the specified identification and the specified classification.

5. The illegal behavior presumption system according to claim 4, characterized in that: The category specifying mechanism specifies the category of the article from a plurality of predetermined categories. The category identifier creating mechanism creates the category identifier based on the plurality of categories.

6. A system for inferring illegal behavior, characterized in that Include: An item information acquisition unit acquires item information including an item image showing the item; An identification specific mechanism is used to identify the object based on the object image; A position information acquisition mechanism, which acquires position information related to the position of the specific mark in the object image; A classification specifying mechanism specifies the classification of the object after performing a color correction process on a boundary box representing the position information in the object image so that the features of the identification part disappear; as well as The inference unit infers illegal behavior related to the article based on the specified identification and the specified classification.

7. A system for presumpting illegal behavior, characterized in that Include: The eigenvalue calculator production organization produces a machine learning model that calculates language eigenvalues ​​in vector form, namely, an eigenvalue calculator, based on the description text of regular items; An item information acquisition agency, which acquires item information related to the item; An identification specific mechanism is used to identify the item based on the item information; A classification specifying mechanism specifies a classification as a type or category of the item based on the item information; as well as The inferring means infers illegal behavior related to the article based on the feature value of the identified identifier calculated by the feature value calculator and the feature value of the identified classification calculated by the feature value calculator.

8. The illegal behavior presumption system according to claim 7, characterized in that: The feature value calculator creating means creates the feature value calculator based on a description of a genuine article that is a legitimate article as a result of estimation by the estimation means.

9. The illegal behavior presumption system according to any one of claims 1 to 8, characterized in that: The illegal behavior estimation system further includes a related data acquisition unit, the related data acquisition unit acquiring related data that associates each of the plurality of identifiers with at least one classification. The inference unit infers illegal behavior related to the article based on the identified identifier, the identified classification, and the associated data.

10. The illegal behavior presumption system according to any one of claims 1 to 8, characterized in that: The items described are commodities, The item information is product information related to the product. The identification specifying mechanism specifies the identification of the commodity based on the commodity information. The category specifying unit specifies the category of the commodity based on the commodity information. The presumption mechanism presumes illegal behavior related to the product.

11. A method for presumpting illegal behavior, characterized in that The computer performs the following steps: a marker identifier making step, by acquiring teaching data that uses the image or its feature value as input and the marker shown in the image as output based on an image showing a marker to be identified, and causing the marker identifier to learn in machine learning based on the acquired teaching data, thereby making the marker identifier; an item information acquisition step of acquiring item information including an item image showing the item; an identification specifying step of specifying the identification of the object based on the object image and the identification identifier; A classification specifying step of specifying the classification of the item based on the item information; as well as The inference step is to infer an illegal act related to the article based on the identified identification and the identified classification.

12. A method for presumpting illegal behavior, characterized in that The computer performs the following steps: A classification identifier making step, comprising: obtaining teaching data that uses an image or a feature value of an object showing a classification to be identified as an input and a classification shown by the image as an output, and making the classification identifier learn in machine learning based on the obtained teaching data, thereby making the classification identifier; an item information acquisition step of acquiring item information including an item image showing the item; an identification step of identifying the item based on the item information; A classification specifying step, specifying the classification of the object based on the object image and the classification identifier; as well as The inference step is to infer an illegal act related to the article based on the identified identification and the identified classification.

13. A method for presumpting illegal behavior, characterized in that The computer performs the following steps: an item information acquisition step of acquiring item information including an item image showing the item; an identification step of identifying the object based on the object image; a position information acquisition step, acquiring position information related to the position of the identified mark in the object image; a classification specifying step, wherein a color correction process is performed on a boundary box representing the position information in the object image so that the features of the identification part disappear, and then the classification of the object is specified; as well as The inference step is to infer an illegal act related to the article based on the identified identification and the identified classification.

14. A method for presumpting illegal behavior, characterized in that The computer performs the following steps: The steps of making an eigenvalue calculator are as follows: based on the description of regular items, a machine learning model, i.e., an eigenvalue calculator, is made to calculate language eigenvalues ​​in vector form; An item information acquisition step, acquiring item information related to the item; an identification step of identifying the item based on the item information; A classification specifying step of specifying a classification as a type or category of the item based on the item information; as well as The inference step is to infer an illegal act related to the article based on the feature value of the identified identifier calculated by the feature value calculator and the feature value of the identified classification calculated by the feature value calculator.

15. A storage medium storing a program for causing a computer to function as: The identification identifier production means produces the identification identifier by acquiring teaching data that uses the image or its feature value as input and the identification shown in the image as output based on an image showing the identification to be recognized, and causing the identification identifier to learn in machine learning based on the acquired teaching data; An item information acquisition unit acquires item information including an item image showing the item; An identification specific mechanism that identifies the identification of the object based on the object image and the identification identifier; A classification specifying mechanism that specifies a classification as a type or category of the item based on the item information; and The inference unit infers illegal behavior related to the article based on the specified identification and the specified classification.

16. A storage medium storing a program for causing a computer to function as: A classification identifier making unit, which makes the classification identifier by learning the classification identifier in machine learning based on the acquired teaching data by acquiring teaching data that takes the image or its feature value as input and the classification shown by the image as output based on an image of a subject showing a classification to be identified; An item information acquisition unit acquires item information including an item image showing the item; An identification specific mechanism is used to identify the item based on the item information; A category specifying mechanism that specifies the category of the object based on the object image and the category identifier; and The inference unit infers illegal behavior related to the article based on the specified identification and the specified classification.

17. A storage medium storing a program for causing a computer to function as: An item information acquisition unit acquires item information including an item image showing the item; An identification specific mechanism is used to identify the object based on the object image; A position information acquisition mechanism, which acquires position information related to the position of the specific mark in the object image; a classification specifying mechanism that specifies the classification of the object after performing a color correction process on a boundary box representing the position information in the object image so that the features of the identification portion disappear; and The inference unit infers illegal behavior related to the article based on the specified identification and the specified classification.

18. A storage medium storing a program for causing a computer to function as: The eigenvalue calculator production organization produces a machine learning model that calculates language eigenvalues ​​in vector form, namely, an eigenvalue calculator, based on the description text of regular items; An item information acquisition agency, which acquires item information related to the item; An identification specific mechanism is used to identify the item based on the item information; A classification specifying mechanism that specifies a classification as a type or category of the item based on the item information; and The inferring means infers illegal behavior related to the article based on the feature value of the identified identifier calculated by the feature value calculator and the feature value of the identified classification calculated by the feature value calculator.

Images