Augmented reality product information system

Abstract

To easily and quickly provide a user with additional information and update information on posted merchandise of a merchandise sales paper medium.SOLUTION: A barcode representing a paper medium identification code is printed on a merchandise sales paper medium 1, and a marker including identification information of merchandise is printed in a merchandise information posting column of the paper medium. The paper medium identification code and attribute information of the marker are registered in association with each other in a first database 6, and the identification information of the merchandise and additional information and / or update information on the merchandise are registered in association with each other for each paper medium identification code in a second database 7. A portable terminal communicates with a management server 8. The portable terminal 10 detects the marker on the basis of the attribute information of the marker to read the identification information of the merchandise after the paper medium identification code is detected. The portable terminal displays an AR image obtained by overlapping corresponding additional information and / or update information on a photographed image of the merchandise information posting column.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a system that uses augmented reality (AR) to provide users with additional and updated information about products published in paper media intended for product sales, such as catalogs and flyers, to their mobile devices. [Background technology]

[0002] Paper media for product sales, such as catalogs and flyers, contain information necessary for users to select products to purchase, but there are certain limitations on the size and number of pages of the paper media for product sales, and the product information listing space defined for each product is very limited.

[0003] In this case, the amount of information that can be printed can be increased by reducing the size of the printed characters; however, in order to ensure readability on the page, characters of a certain size or larger must be used, and therefore the amount of information printed on the page is kept to a minimum.

[0004] However, for some users, the information published in the paper may not be sufficient for selecting a product and additional information may be required. Also, on the seller's side, when the information on the published product is updated after the distribution of the product sales paper medium, there may be cases where the seller wants to provide users with updated information each time, or to provide users with additional information for sales promotion purposes.

[0005] Therefore, in the prior art, there is a system in which a QR code (registered trademark) is attached to each product information section of a product sales paper medium, and a user can obtain additional and updated information about the products listed in the product sales paper medium by reading the QR code with a mobile device and accessing a related website based on the URL indicated by the QR code (see, for example, Patent Document 1, etc.).

[0006] However, with this system, when a user needs to obtain additional or updated information about a product listed in a print product sales medium, the user must always use a mobile device to access the relevant website, which requires time and effort to obtain product information and may hinder the promotion of product sales. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Utility Model Registration No. 3175609 Summary of the Invention [Problem to be solved by the invention]

[0008] Therefore, an object of the present invention is to provide a method for easily and quickly providing users with additional and updated information regarding products listed in a product sales medium. [Means for solving the problem]

[0009] In order to solve the above-mentioned problems, according to the present invention, a product sales paper medium is provided with a plurality of types of product sales paper media, each of which has a product information posting section defined for each product on its paper surface, a one-dimensional or two-dimensional barcode storing a paper medium identification code is printed on each of the plurality of types of product sales paper media, a marker unique to the product sales paper medium including identification information of the product posted in the product information posting section is printed in the product information posting section of the product sales paper medium, the product sales paper medium is further provided with a first database in which the paper medium identification code and attribute information of the marker are registered in association with each other, a second database in which the identification information of the product and additional information and / or updated information related to the product are registered in association with each other for each of the paper medium identification codes, a management server connected to the first and second databases, and a mobile terminal connected to the management server via a communication network, and when the management server receives the paper medium identification code from the mobile terminal, the management server extracts attribute information of the marker corresponding to the paper medium identification code from the first database and transmits the extracted attribute information to the mobile terminal, and then transmits the extracted attribute information of the product from the mobile terminal. When the identification information is received, the additional information and / or update information corresponding to the paper medium identification code and the identification information of the product is extracted from the second database and transmitted to the mobile terminal, the mobile terminal having an imaging unit that photographs the product sales paper medium, an image display unit that displays an image photographed by the imaging unit, a communication unit that performs data communication with the management server, and a first detection unit that reads the one-dimensional or two-dimensional barcode of the product sales paper medium and detects the paper medium identification code in the barcode, the communication unit transmits the paper medium identification code detected by the first detection unit to the management server and receives attribute information of the corresponding marker from the management server, the mobile terminal further has an attribute information storage unit in which the attribute information of the marker received by the communication unit is stored, and a second detection unit that detects the marker in the image photographed by the imaging unit based on the attribute information of the marker stored in the attribute information storage unit and reads the identification information of the product in the marker, the communication unit transmits the identification information of the product read by the second detection unit to the management server,The system for providing product information using augmented reality is characterized in that the system receives the corresponding additional information and / or updated information from the management server, and when the second detection unit detects the marker, the image display unit displays an AR image in which the corresponding additional information and / or updated information received by the communication unit is superimposed on the image captured by the imaging unit.

[0010] According to a preferred embodiment of the present invention, the product identification information is a product identification code, the marker comprises a reference mark consisting of a different type of figure or symbol from the figures and symbols used in printing the product information column of the product sales paper medium, and the product identification code adjacent to the right side of the reference mark, and the attribute information of the marker is image data of the reference mark and the number of digits of the product identification code.

[0011] According to another preferred embodiment of the present invention, the product identification information is a product name and / or a product identification code, the marker consists of the product name and / or the product identification code printed in a font different from the font of characters used to print the product information column of the product sales paper medium, and the attribute information of the marker is information on the font of the product name or the product identification code.

[0012] According to yet another preferred embodiment of the present invention, the product identification information is a product name and / or a product identification code, the marker comprises a recognition area having a predetermined size and shape and a color different from the surrounding color, and a product name and / or a product identification code printed in the recognition area, and the attribute information of the marker is the size and shape of the recognition area.

[0013] According to yet another preferred embodiment of the present invention, the product sales paper medium comprises a catalog and / or a flyer. Effect of the Invention

[0014] Thus, according to the present invention, when a user identifies the type of product sales medium by reading the one-dimensional or two-dimensional barcode on the product sales paper medium with a mobile device and then photographs the product information section of the product he or she wishes to purchase on the paper page of the product sales paper medium with the mobile device, additional and / or updated information about the product registered in the second database is displayed on the mobile device superimposed on the photographed image of the product information section.

[0015] This allows users to easily and quickly obtain additional and / or updated information regarding products listed in the product sales paper medium without having to access a website. [Brief description of the drawings]

[0016] [Figure 1] 1 is a block diagram showing a configuration of a product information providing system using augmented reality according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram showing an example of a product sales paper medium in the system of FIG. 1. [Diagram 3] FIG. 2 is a diagram showing an example of a product sales paper medium in the system of FIG. 1. [Figure 4] FIG. 2A is a diagram showing the structure of a first database in the system of FIG. 1, and FIG. 2B is a diagram showing the structure of a second database in the system of FIG. [Diagram 5] FIG. 2 is a diagram showing an example of a screen displayed on a display of a mobile terminal in the system of FIG. [Figure 6] FIG. 13 is a diagram showing a modified example of the marker. [Figure 7] FIG. 13 is a diagram showing a modified example of the marker. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the configuration of the present invention will be described based on preferred embodiments with reference to the accompanying drawings. FIG. 1 is a block diagram showing the configuration of a merchandise information providing system according to one embodiment of the present invention, and FIG. 2 and FIG. 3 are plan views showing examples of merchandise sales paper media in the merchandise information providing system of FIG.

[0018] 1 to 3, a product information provision system using augmented reality according to the present invention includes a plurality of types of product sales paper media 1, each of which has a product information column 2 defined for each product on its surface. In this embodiment, the multiple types of product sales paper media 1 are catalogs and / or flyers.

[0019] Each of the multiple types of product sales paper media 1 has a two-dimensional barcode 3 containing a paper medium identification code printed on the cover, and each product information section 2 has a marker 4 printed in it that is unique to the product sales paper medium 1 and contains identification information 5 for the product listed in that product information section 2 (in this embodiment, a product identification code).

[0020] In this case, a one-dimensional barcode can be used instead of the two-dimensional barcode 3 as the barcode for storing the paper medium identification code.

[0021] In this embodiment, the marker 4 consists of a reference mark (in the illustrated product sales paper medium 1, a "#") consisting of a figure or symbol different from the figures and symbols used to print the product information column of the product sales paper medium 1, and a product identification code 5 (in the illustrated product sales paper medium 1, a four-digit number) adjacent to the right of the reference mark "#".

[0022] The product information providing system of the present invention further includes a first database 6 in which the paper medium identification code and the attribute information of the marker 4 are registered in association with each other. In this embodiment, the attribute information of the marker 4 is made up of image data of the reference mark and the number of digits of the product identification code 5.

[0023] FIG. 4A is a diagram showing the structure of the first database 6. As shown in FIG. As shown in FIG. 4A, in this embodiment, the first database 6 includes a plurality of records 17 each of which is provided for each paper medium identification code. a , 17 n It has. And for each record 17 a , , 17 n There are 6 fields 18 a ~18 f The first field is "Product Code" 18 a The paper medium identification code is stored in the first field, "medium name" 18 b The name of the paper medium is stored in the third field, "Company name" 18 c The name of the company that issued the paper medium is stored in the fourth field, "Standard Mark" 18 d The image data of the reference mark of marker 4 is stored in the 5th field, "Number of digits" 18 e The number of digits of the product identification code 5 is stored in the 6th field "Release date" 18 f The publication date of the paper version is stored in .

[0024] The product information providing system of the present invention further includes a second database 7 in which a product identification code (product identification information) 5 and additional information and / or updated information relating to the product are registered in association with each other for each paper medium identification code.

[0025] FIG. 4B is a diagram showing the structure of the second database 8. As shown in FIG. As shown in FIG. 4B, the second database 8 includes a plurality of records 19 each of which is provided for each paper medium identification code and product identification code 5. a , 19 b , , 19 n-1 , 19 n It has. And each record 19 a , 19 b , , 19 n-1 , 19 n There are four fields 20 a ~20 d The first field is "media code" 20 aThe paper medium identification code is stored in the second field "Product Code" 20 b The product identification code is stored in the third field "Display 1" 20 c The first additional and / or updated information about the product is stored in the fourth field “Display 2” 20 d A second additional and / or updated information about the product is stored in the

[0026] The product information providing system of the present invention further includes a management server 8 connected to the first and second databases 6, 7, and a mobile terminal 10 connected to the management server 8 via a communication network 9.

[0027] When the management server 8 receives a paper medium identification code from the mobile terminal 10, it extracts attribute information of the marker 4 corresponding to the paper medium identification code from the first database 6 and transmits it to the mobile terminal 10. Then, when the management server 8 receives a product identification code (product identification information) 5 from the mobile terminal 10, it extracts additional information and / or updated information (first and second additional information and / or updated information) corresponding to the paper medium identification code and the product identification code (product identification information) 5 from the second database 7 and transmits it to the mobile terminal 10.

[0028] The mobile terminal 10 has an imaging unit 11 that photographs the product sales paper medium 1, an image display unit 12 that includes a display and displays the image photographed by the imaging unit 11, a communication unit 13 that performs data communication with the management server 8, and a first detection unit 14 that reads the two-dimensional barcode 3 on the product sales paper medium 1 and detects the paper medium identification code in the two-dimensional barcode 3. The first detection unit 14 is made up of a barcode reader.

[0029] The communication unit 11 transmits the paper medium identification code detected by the first detection unit 14 to the management server 8 and receives attribute information of the marker 4 from the management server 8 .

[0030] The mobile terminal 10 further has an attribute information storage unit 15 in which attribute information of the marker 4 received by the communication unit 11 is stored, and a second detection unit 16 that detects the marker 4 in the image captured by the imaging unit 11 based on the attribute information of the marker 4 stored in the attribute information storage unit 15, and reads the product identification code (product identification information) 5 in the marker 4.

[0031] Detection by the second detection unit 16 is performed by using pattern matching image processing to detect an image (position) in the image captured by the imaging unit 11 that matches the image of the reference mark stored in the attribute information storage unit 15, and then using OCR (optical character recognition) to read a character string equivalent to the number of digits of the product identification code stored in the attribute information storage unit 15 in the area adjacent to the right of the detected image.

[0032] The communication unit 11 transmits the product identification code (product identification information) 5 read by the second detection unit 16 to the management server 8, and receives corresponding additional information and / or updated information from the management server 8. Furthermore, when the second detection unit 16 detects the marker 4, the image display unit 12 displays an AR image in which the corresponding additional information and / or update information received by the communication unit 11 is superimposed on the image captured by the imaging unit 11.

[0033] Next, the operation of the product information providing system of the present invention will be described. First, when a user reads the two-dimensional barcode 3 on the product sales paper medium 1 using the mobile terminal 10, the first detection unit 14 of the mobile terminal 10 detects the paper medium identification code ("001-0001" in the illustrated embodiment) in the two-dimensional barcode 3, and the paper medium identification code "001-0001" is transmitted from the mobile terminal 10 to the management server 8.

[0034] Next, the management server 8 retrieves attribute information of the marker 4 corresponding to the paper medium identification code “001-0001” from the first database 6 (record 17a; fourth field “reference mark” 18 d The image data of the "#" and the 5th field "Number of digits" 18 eThe mobile terminal 10 extracts the attribute information of the marker 4 from the mobile terminal 10 and transmits it to the mobile terminal 10. The attribute information of the marker 4 received by the mobile terminal 10 is stored in the mobile terminal 10.

[0035] Then, the user photographs an interesting product information section 2 on the page of the product sales paper medium 1 with the mobile terminal 10, for example, as shown in Fig. 5. In the situation shown in Fig. 5, two product information sections 2, and therefore two markers 4, are photographed simultaneously. At this time, the second detection unit 16 of the mobile terminal 10 detects the reference mark "#" present in each of the two product listing columns 2, as well as the product identification code "0002" associated with one of the reference marks "#", and the product identification code "0005" associated with the other reference mark "#", based on the attribute information of the marker 4, and the product identification codes "0002" and "0005" are transmitted from the mobile terminal 10 to the management server 8.

[0036] Furthermore, the management server 8 retrieves additional information and / or updated information (record 19) corresponding to the paper medium identification code “001-0001” and the product identification code “0002” from the second database 7. n-1 ;Third field "Display 1" 20 c and the fourth field "Display 2" 20 d (each of the above information), paper medium identification code "001-0001" and product identification code "000 5 " Additional and / or updated information (Record 19 n ;Third field "Display 1" 20 c and the fourth field "Display 2" 20 d The information on the above items is extracted and transmitted to the mobile terminal 10.

[0037] Then, as shown in FIG. 5, an AR image is displayed on the display 22 of the mobile terminal 10 in which the corresponding additional information and / or updated information received by the mobile terminal 10 is superimposed on the image of the product sales paper medium 1 already displayed.

[0038] Thus, according to the present invention, a user identifies the type of product sales medium 1 by reading the two-dimensional barcode 3 on the product sales paper medium 1 with the mobile terminal 10, and then photographs the product information section of the product he or she wishes to purchase on the page of the product sales paper medium 1 with the mobile terminal 10. An AR image is then displayed on the display 22 of the mobile terminal 10, in which additional information and / or updated information about the product is superimposed on the photographed image of the product information section.

[0039] This allows users to easily and quickly obtain additional and / or updated information about the products listed on the product sales paper medium 1 without having to access a website. On the other hand, if the seller has information that they wish to add to the information published on the product sales paper medium 1, or if they update the published information after distributing the product sales paper medium 1, they can make the additional and updated information convenient for users by registering it in the second database 7.

[0040] Although a preferred embodiment of the present invention has been described above, the configuration of the present invention is not limited to the above embodiment, and it goes without saying that a person skilled in the art can devise various modifications within the scope of the configuration described in the claims of this application.

[0041] For example, in the above embodiment, the product identification information is a product identification code, and marker 4 consists of a reference mark consisting of a different type of figure or symbol from the figures and symbols used to print product sales paper medium 1, and a product identification code adjacent to the right of the reference mark, but marker 4 may have any configuration as long as it is a marker unique to product sales paper medium 1 containing the product identification information listed in product information listing section 2.

[0042] According to another embodiment of the present invention, the product identification information is a product name (and / or product identification code) 23, and the marker 4' consists of the product name (and / or product identification code) 23 printed in a font (in this embodiment, HC Soei Kaku Pop) different from the font of characters used to print the product sales paper medium 1 (in this embodiment, a mixture of Gothic and Mincho fonts), as shown in FIG. 6. In this embodiment, the attribute information of the marker 4 is font information of the printed product name (and / or product identification code) 23.

[0043] According to yet another embodiment of the present invention, the identification information of the product is a product identification code (and / or product name) 24, and the marker 4″, as shown in FIG. 7, comprises a recognition area 25 (a white rectangular area of ​​a predetermined size in this embodiment) having a predetermined size and shape and a color different from the surrounding color, and the product identification code (and / or product name) 24 printed in the recognition area 25. In this embodiment, the attribute information of the marker 4 is the size and shape of the recognition area 25 . [Explanation of symbols]

[0044] 1. Product sales paper media 2 Product information section 3. 2D barcode 4, 4', 4” Markers 5 Product Identification Code 6. The First Database 7. The second database 8 Management Server 9. Communication Networks 10 Mobile devices 11 Imaging unit 12 Image display section 13. Communications Department 14 First detection unit 15 Attribute information storage section 16 Second detector 17 a ~17 n record 18 a ~18d field 19 a ~19 n record 20 a ~20 d field 21 Product Information Images 22 Display 23 Product name 24 Product Identification Code 25 Recognition Area

Claims

1. A plurality of types of product sales paper media are provided, each of which has a product information column defined for each product on its surface; A one-dimensional or two-dimensional barcode storing a paper medium identification code is printed on each of the multiple types of product sales paper media, and a marker unique to the product sales paper medium including identification information of the product listed in the product information listing section is printed in the product information listing section of the product sales paper medium, and further a first database in which the paper medium identification code and attribute information of the marker are registered in association with each other; a second database in which identification information of the product and additional information and / or updated information related to the product are registered in association with each other for each of the paper medium identification codes; a management server connected to the first and second databases; a mobile terminal connected to the management server via a communication network; The management server includes: When the paper medium identification code is received from the mobile terminal, attribute information of the marker corresponding to the paper medium identification code is extracted from the first database and transmitted to the mobile terminal. Thereafter, when the product identification information is received from the mobile terminal, the paper medium identification code already received from the mobile terminal is also used to extract the additional information and / or updated information corresponding to the paper medium identification code and the product identification information from the second database and transmitted to the mobile terminal. The mobile terminal includes: An imaging unit that captures an image of the product sales paper medium; an image display unit that displays an image captured by the imaging unit; A communication unit that performs data communication with the management server; a first detection unit that reads the one-dimensional or two-dimensional barcode of the product sales paper medium and detects the paper medium identification code in the barcode; the communication unit transmits the paper medium identification code detected by the first detection unit to the management server, and receives attribute information of the corresponding marker from the management server; The mobile terminal further includes: an attribute information storage unit in which the attribute information of the marker received by the communication unit is stored; a second detection unit that detects the marker in the image captured by the imaging unit based on the attribute information of the marker stored in the attribute information storage unit, and reads identification information of the product in the marker; the communication unit transmits the identification information of the product read by the second detection unit to the management server, and receives the corresponding additional information and / or updated information from the management server; A product information provision system using augmented reality, characterized in that when the second detection unit detects the marker, the image display unit displays an AR image in which the corresponding additional information and / or updated information received by the communication unit is superimposed on the image captured by the imaging unit.

2. The identification information of the product is a product identification code, The marker is A reference mark consisting of a figure or symbol of a type different from the figure and symbol used in printing the product information column of the product sales paper medium; the product identification code adjacent to the right side of the fiducial mark; 2. The product information providing system using augmented reality according to claim 1, wherein the attribute information of the marker is image data of the reference mark and the number of digits of the product identification code.

3. The product identification information is a product name and / or a product identification code, the marker is the product name or the product identification code printed in a font different from a font of characters used to print the product information column of the product sales paper medium, The product information providing system using augmented reality according to claim 1 , wherein the attribute information of the marker is information on the font of the product name and / or the product identification code.

4. The product identification information is a product name and / or a product identification code, The marker is A recognition area having a predetermined size and shape and a color different from the color of its surroundings; the product name and / or the product identification code printed in the recognition area; The product information providing system using augmented reality according to claim 1 , wherein the attribute information of the marker is the size and the shape of the recognition area.

5. 5. The product information providing system using augmented reality according to claim 1, wherein the product sales paper medium is a catalog and / or a flyer.

Images