Processing apparatus, processing method, and processing program including an image of a beverage can

Abstract

The present disclosure obtains a processing device, a processing method, and a processing program that include an image of a suitable (improved) beverage can. A processing device that includes an image of a beverage can according to an aspect of the present disclosure has a first straight line acquisition section, a second straight line acquisition section, and an angle acquisition section. The first straight line acquisition section acquires a first straight line from the image including the beverage can, the first straight line linking a center point of an upper surface and a center point of a drinking port or a tip point of the drinking port that is farthest from the center point of the upper surface. The second straight line acquisition section acquires a second straight line from the image including the beverage can, the second straight line linking the center point of the upper surface and a point at which a straight line extending perpendicularly from a reference point of an identification section of a side toward a plane including the upper surface intersects the plane including the upper surface. The angle acquisition section acquires an angle formed by the first straight line and the second straight line.

Description

Technical Field

[0001] This disclosure relates to processing apparatus, processing methods, and processing procedures for images containing beverage cans. Background Technology

[0002] Patent Document 1 discloses a drunk driving prevention device comprising an individual identification information reading device, a location estimation device, and a drunk driving suppression control device. The individual identification information reading device reads individual identification information from a container of alcoholic beverage inside a vehicle. The location estimation device estimates the location of the alcoholic beverage inside the vehicle based on the output of the individual identification information reading device. The drunk driving suppression control device performs control measures to suppress drunk driving based on the location of the alcoholic beverage estimated by the location estimation device. Furthermore, Patent Document 1 exemplifies an RFID reader as a specific example of a device for reading the individual identification information of alcoholic beverages. Moreover, Patent Document 1 also discloses that the device for reading the individual identification information of alcoholic beverages is not limited to a wireless communication-based device; for example, it could be a barcode scanner, and the alcoholic beverage could be read through image recognition.

[0003] Patent Document 2 describes a notification control system comprising a user information receiving device, a store information receiving device, a matching device, and a purchase information notification device. The user information receiving device receives user information related to a beverage from a user. The store information receiving device receives store information related to the purchase of a beverage from a physical or virtual store. The matching device matches user information with store information based on the attributes of the beverage. The purchase information notification device notifies the user of purchase information based on the user information and store information matched by the matching device. Furthermore, Patent Document 2 also describes the matching device performing image recognition of the user information image data to determine the attributes of the beverage. In this case, known image recognition technologies (OCR technology, shape recognition technology, etc.) can also be used to perform image recognition of the user information image data to determine the attributes of the beverage.

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 2009-258855

[0007] Patent Document 2: Japanese Patent Application Publication No. 2017-091343 Summary of the Invention

[0008] The problem that the invention aims to solve

[0009] However, from the perspective of processing images containing suitable (improved) beverage cans, there is room for improvement in the prior art, including patent documents 1 and 2.

[0010] Therefore, one of the objectives of this disclosure is to obtain a suitable (improved) processing apparatus, processing method, and processing procedure for images containing beverage cans.

[0011] Technical solutions for solving the problem

[0012] According to one aspect of the present disclosure, an image processing apparatus containing a beverage can includes: an upper surface that is generally circular in plan view; a lower surface that is generally circular in plan view; a side surface that is cylindrical and generally annular in plan view, connecting the upper surface and the lower surface; a drinking spout disposed between the center point and the outer periphery of the upper surface; and a marking portion disposed on the side surface. The image processing apparatus is characterized by having: an image acquisition unit that acquires an image containing the beverage can captured by a camera unit of a user terminal and transmitted by an image transmission unit of the user terminal. Image; a first straight line acquisition unit that acquires a first straight line from the image, the first straight line connecting the center point of the upper surface and the center point of the drinking spout or the tip of the drinking spout furthest from the center point of the upper surface; a second straight line acquisition unit that acquires a second straight line from the image, the second straight line connecting the center point of the upper surface and the point where a straight line extending perpendicularly from the reference point of the marking portion on the side surface toward the plane containing the upper surface intersects the plane containing the upper surface; and an angle acquisition unit that acquires the angle formed by the first straight line and the second straight line.

[0013] The image processing apparatus containing a beverage can disclosed herein is characterized by comprising: an image acquisition unit that acquires an image containing a beverage can; an angle calculation unit that parses the image and calculates the angle formed by a first straight line and a second straight line set based on the contour features of the beverage can; and an angle output unit that outputs the angle formed by the first straight line and the second straight line.

[0014] The contour feature may include at least one of the following: a drinking spout on the upper surface of the beverage can, a central portion on the upper surface of the beverage can, and a display portion on the side of the beverage can.

[0015] The first straight line can be a straight line connecting the tip of the drinking spout on the upper surface of the beverage can to the center of the upper surface of the beverage can. The second straight line is a straight line connecting the point of intersection of a straight line including a display part on the side of the beverage can and extending in the height direction of the beverage can with the upper surface of the beverage can and the center of the upper surface of the beverage can.

[0016] It may also include a judgment unit that analyzes the image and determines whether the beverage can is a specified brand, whether the beverage can is a specified capacity, and whether the image is an existing image that lacks originality.

[0017] The image acquisition unit can acquire multiple images from the same user terminal. The angle calculation unit calculates the angle formed by the first straight line and the second straight line for the multiple images. The angle output unit outputs the angle formed by the first straight line and the second straight line for the multiple images. It may also have a comparison unit that compares whether the angle formed by the first straight line and the second straight line is within a specified range for the multiple images.

[0018] It may also have a notification unit that notifies the user terminal of the above content when the angle formed by the first straight line and the second straight line is within a specified range for the plurality of images.

[0019] It may also have a discount granting unit, which, for the plurality of images, grants the user terminal a discount corresponding to the number of the plurality of images when the angle formed by the first straight line and the second straight line is not within a specified range.

[0020] According to one aspect of this disclosure, a processing apparatus for an image of a beverage can includes: an upper surface that is circular in plan view; a lower surface that is circular in plan view; a side surface that is cylindrical and annular in plan view, connecting the upper and lower surfaces; a drinking spout disposed on one side away from the center point of the upper surface; a pull tab disposed on the other side away from the center point of the upper surface for opening the drinking spout; and a display unit disposed on the side surface and including the manufacturer and trade name of the beverage can. The processing apparatus for the image of the beverage can is characterized by having: an image acquisition unit that acquires images captured by a camera unit of a user terminal and processed by... The image transmitting unit of the user terminal transmits an image containing the beverage can; and the angle calculation unit calculates the angle formed by the first straight line and the second straight line when a first straight line and a second straight line are defined, wherein the first straight line connects the tip of the drinking spout on the upper surface of the beverage can that is farthest from the center point of the upper surface of the beverage can and the center point of the upper surface of the beverage can, and the second straight line connects the point where a straight line including a point of the display part on the side of the beverage can and extending perpendicularly toward the upper surface of the beverage can intersects the upper surface of the beverage can and the center point of the upper surface of the beverage can.

[0021] Invention Effects

[0022] According to one aspect of this disclosure, a processing apparatus, processing method, and processing procedure are available for obtaining an image containing a suitable (modified) beverage can. Attached Figure Description

[0023] Figure 1 This is a diagram illustrating an example of the structure of a system containing a processing apparatus for an image of a beverage can.

[0024] Figure 2 A diagram illustrating an example of the structure of a user terminal.

[0025] Figure 3 This is an example diagram showing an image containing a beverage can.

[0026] Figure 4A , Figure 4B , Figure 4C A diagram illustrating an example of the concept of a beverage can image parsing application service based on an image containing a beverage can.

[0027] Figure 5 A diagram illustrating an example of the structure of an image processing apparatus.

[0028] Figure 6 A diagram showing an example of the structure of the angle calculation unit.

[0029] Figure 7 A diagram illustrating the setting of the first and second straight lines and an example of the angle between the first and second straight lines.

[0030] Figure 8 A diagram illustrating an example of the concept of a beverage can image parsing application service based on an image containing a beverage can.

[0031] Figure 9 This is a diagram illustrating an example of a process for processing an image containing a beverage can.

[0032] Figure 10 This diagram illustrates an example of the functional configuration of a user terminal and a processing device containing an image of a beverage can.

[0033] Figure 11 This diagram illustrates an example of the hardware configuration of a user terminal and a processing device containing an image of a beverage can. Detailed Implementation

[0034] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0035] Figure 1This diagram illustrates an example of the structure of a system 1 containing a processing apparatus 20 for images of beverage cans. "Processing apparatus for images of beverage cans" can also be replaced with "processing system for images of beverage cans" (both are synonymous). Furthermore, the processing method and program for images of beverage cans are implemented by having the computer (CPU: Central Processing Unit) included in system 1 (user terminal 10, processing apparatus 20 for images of beverage cans) execute various processing steps.

[0036] like Figure 1 As shown, system 1 includes a user terminal 10, an image processing device (hereinafter referred to as "image processing device") 20 containing an image of a beverage can, and a network 30 connecting the user terminal 10 and the image processing device 20. Figure 1 In this drawing, the user terminal 10 and the image processing device 20 are simplified into a single structure for ease of illustration. In reality, whether as a hardware structure or a software structure, the user terminal 10 and the image processing device 20 possess advantages over... Figure 1 A more complex structure can be shown. For example, the image processing device 20 can also be configured to include multiple servers, databases, etc.

[0037] System 1 can be understood as either a structure including user terminal 10 and image processing device 20, or a structure that does not include user terminal 10 but only image processing device 20. That is to say, in System 1, user terminal 10 is not a necessary component (the image containing the beverage can is input to image processing device 20, as described later).

[0038] User terminal 10 is, for example, a portable terminal (mobile communication terminal) such as a mobile phone, smartphone, or tablet. In other words, user terminal 10 in this specification is a communication device capable of various forms of communication with image processing device 20. User terminal 10 can, for example, communicate via wired and / or wireless (e.g., Long Term Evolution (LTE), 5G) communication devices. th Generation New Radio (5G NR), Wi-Fi (registered trademark), etc., are used to communicate with networks (such as the Internet). User terminal 10 executes a browser and various other applications (especially the beverage can image parsing application service described later) through user operation.

[0039] The image processing device 20 can be any communication device capable of various communications with the user terminal 10, or it can be replaced by equipment, circuits, etc. In particular, the image processing device 20 communicates with the user terminal 10 to parse the image containing the beverage can obtained (captured) by the user terminal 10 and provide various application services based on the parsing results (beverage can image parsing application service).

[0040] Figure 2 This diagram illustrates an example of the structure of a user terminal 10. The user terminal 10 includes a camera unit (image acquisition unit) 11, an image transmission unit 12, and a beverage can image analysis and application unit 13. In addition, the user terminal 10 also includes… Figure 2 In addition to the components shown, it also has components typically found on a user terminal, such as a display, speaker, and microphone.

[0041] The camera unit 11 is a camera system mounted in the user terminal 10, which captures (acquires) images containing beverage cans. The image transmission unit 12 transmits the images containing beverage cans captured by the camera unit 11 to the image processing device 20. The beverage can image analysis application unit 13 performs various processes for utilizing the beverage can image analysis application service based on the images containing beverage cans transmitted from the image transmission unit 12 to the image processing device 20.

[0042] For example, when the camera of the imaging unit 11 is activated, it can also determine whether the beverage can in the image received by the imaging element (CCD: Charge Coupled Device) is a specified brand before taking a photo. To achieve a more comfortable user experience (UX) than taking a separate photo, the beverage can determination process can be segmented. In the determination process, for example, it can analyze whether the image received by the imaging element contains the logo of the specified brand of the beverage can or the top surface of the beverage can, analyze the outline shape and appearance design of the beverage can, and utilize them. At this time, it can also calculate the probability (reliability score) that the beverage can received by the imaging element is a specified brand, and perform the photo taking if this probability is above a specified value (e.g., above 50%). Here, there is a degree of freedom in determining whether to perform the photo taking based on the specified value, and various design changes can be made. Of course, a 1 / 0 flag can also be used to detect whether the beverage can received by the imaging element is a specified brand.

[0043] Figure 3This diagram illustrates an example of an image containing a beverage can. The beverage can is, for example, a container holding any beverage such as alcoholic beverages, mineral water, carbonated water, soft drinks, corn chowder, etc. The beverage can has an upper surface that is generally circular in plan view, a lower surface that is also generally circular in plan view, and a cylindrical side that is generally annular in plan view and connects the upper and lower surfaces. On the upper surface of the beverage can, a drinking spout and a pull tab for opening the drinking spout are provided on one side and the other side, separated by a center point. Figure 3 The drinking spout is located to the left of the center point on the upper surface of the beverage can, and the pull tab is located to the right of the center point. The drinking spout and pull tab are respectively positioned between the center point and the outer perimeter of the upper surface of the beverage can. On the side of the beverage can, the manufacturer and product name (brand) are displayed as identification (shown). Figure 3 In the diagram, XYZ is drawn as the manufacturer of the beverage cans, and ABCDE is drawn as the trade name of the beverage cans. Figure 3 In China, for ease of drawing, the labeling (manufacturer and product name) on the side of the beverage can is drawn in a normal font. However, in reality, the labeling (manufacturer and product name) on the side of the beverage can is photographed in a distorted manner according to the curvature of the beverage can.

[0044] Figure 3 The shape of the beverage can shown is just an example; other types and shapes of beverage cans can be used as objects. Figure 3 When viewed from above, the top, bottom, and sides of a beverage can appear as circles or rings of the same diameter. Conversely, the top and bottom surfaces can also be circles of different diameters, or the diameter of the sides can vary depending on their height. For example, a beverage can can also resemble a beer keg (a barrel with a central bulge), where the center bulges out more than the top and bottom surfaces. Furthermore, the top, bottom, and sides do not need to be perfectly circular or ring-shaped when viewed from above; they can be slightly deviated from perfect circles or rings. In this sense, the phrases "approximately circular when viewed from above" and "approximately ring-shaped when viewed from above" are used.

[0045] Figures 4A to 4C This diagram illustrates an example of a beverage can image analysis application service concept based on images of beverage cans. The beverage can image analysis application service is a service that uses the experience of capturing beverage cans with a camera to create brand fans while providing benefits (affordable experiences). To access the beverage can image analysis application service, users need to express their willingness to participate via user terminal 10, based on user confirmation and agreement to the usage rules and privacy policy. Furthermore, the beverage can image analysis application service sets the activity name, recruitment period, target products (type and size of beverage cans), prizes, number of winners, and required points.

[0046] Figure 4A In this process, the camera unit 11 of the user terminal 10 captures (scans) the user's empty beverage can and sends the captured image to the image processing device 20. Figure 4A The taking and sending of images of beverage cans can also be limited by time and frequency (e.g., once a day).

[0047] Figure 4B In China, through Figure 4A If you take and send a certain number of photos of beverage cans, you will be automatically eligible to apply for original merchandise (prizes). Figure 4B In the example, Figure 4A Taking and submitting images of beverage cans is converted into 1 point. Every 5 points accumulated automatically qualify for an original merchandise (prize). The automatic application for every 5 points can also be limited in time and frequency (e.g., a maximum of 10 applications during the event period).

[0048] Figure 4C In the game, if a user wins an automatic application for every 5 points, they will receive original merchandise. Figure 4C In the example shown is a portable terminal case (original smartphone case) with the manufacturer (XYZ) and product name (ABCDE) of the beverage can drawn on it.

[0049] Since the aforementioned beverage can image analysis application service is used to promote beverage cans, it is assumed that, by its very nature, the beverage cans involved in the images sent to the image processing device 20 are not the same (as a premise). That is, it is assumed that each of the prescribed points (e.g., 5 points) required for the automatic application of original peripherals (1 point) cannot be an image of the same beverage can taken multiple times or an image obtained from the Internet, but rather is taken each time the user drinks from a beverage can containing the contents (as a premise). Furthermore, it is assumed that the beverage cans containing the contents that the user drinks were purchased, for example, by the user himself, the user's family, cohabitants, friends who drank the contents of the beverage cans with the user, colleagues, etc. (as a premise).

[0050] The processing apparatus, method, and program for processing images containing beverage cans disclosed herein possess the following constituent elements: by processing (analyzing) the beverage cans contained in the image from the perspective of the aforementioned assumptions and premises as a foundation, it is possible to confirm whether the beverage cans contained in the image are identical, thereby obtaining a "purchase certificate" for the beverage can. For example, if a highly reliable purchase certificate can be obtained smoothly, it is easy to establish a loyalty program even for in-store purchases.

[0051] (loyalty program; a marketing strategy where companies offer preferential treatment to "high-value customers" who use services or purchase goods frequently), and can elevate digital promotion to customer relationship management (CRP) for marketing purposes to increase brand loyalty.

[0052] Here are some of the factors that contribute to the difficulty of obtaining a smooth and reliable purchase confirmation.

[0053] (1) Purchase proof should take into account complaint response and fraud.

[0054] (2) Postcards are troublesome to circulate and not widely distributed, so they are often lost.

[0055] (3) The number of consecutive proofs (series) is enormous due to the production line capacity of the factory, the issuance of barcodes, and the printing of stickers.

[0056] (4) Receipts are cumbersome and time-consuming, and the cost of correct verification is high, resulting in unreasonable loopholes. In the past, there have been examples of organized receipt purchase fraud in promotional activities (date alteration, folding for different payments, alteration of background color, stacking of multiple receipts).

[0057] (5) Electronic payments such as “○○Pay”, which are less prone to errors, suffer from a lack of flexibility due to payment restrictions, retail limitations, and platform (basic operator) dominance.

[0058] Thus, there is a need for a mechanism led (self-developed) by the beverage can manufacturer that processes (analyzes) the beverage cans contained in an image to confirm whether the beverage cans in the image are identical, thereby obtaining proof of purchase for the beverage can. Based on this concern, the inventors have achieved a processing apparatus, method, and program for images containing beverage cans that have such a mechanism. This mechanism is based on the possibility of so-called "pseudo-proof of purchase," which refers to the ability to shift the perception of active proof of purchase from a purely illogical to a purely virtuous one, including allowing for flexibility in assigning certain points as long as obvious irrationality or fraud can be detected, and without necessarily making rigorous purchase judgments.

[0059] Figure 5 This figure shows an example of the structure of an image processing apparatus 20. The image processing apparatus 20 includes an image input unit (image acquisition unit) 21, a first determination unit 22, a second determination unit 23, a third determination unit 24, an angle calculation unit 25, an angle output unit 26, a comparison unit 27, a notification unit 28, and a discount assignment unit 29.

[0060] The image input unit (image acquisition unit) 21 inputs (acquires) an image containing a beverage can, captured by the camera unit 11 of the user terminal 10 and transmitted from the image transmission unit 12. The image input unit (image acquisition unit) 21 also inputs (acquires) multiple images from the same user terminal 10.

[0061] The first determination unit 22 analyzes the image input to the image input unit 21 and determines whether the beverage can contained in the image is a specified brand. This determination can also be the same as the process by which the camera unit 11 of the user terminal 10 determines whether to photograph the beverage can. The first determination unit 22 can also calculate the probability (reliability score) that the beverage can contained in the image input to the image input unit 21 is a specified brand.

[0062] The second determination unit 23 analyzes the image input to the image input unit 21 and determines whether the beverage can contained in the image has a specified capacity. The second determination unit 23 can also determine whether the beverage can contained in the image input to the image input unit 21 is a 500ml can, a 350ml can, or a different capacity. The second determination unit 23 can also calculate the probability (reliability score) that the beverage can contained in the image input to the image input unit 21 is a 500ml can, a 350ml can, or a different capacity.

[0063] The third determination unit 24 analyzes the image input to the image input unit 21 and determines whether the image is an existing image lacking originality. The third determination unit 24 detects whether the image input to the image input unit 21 is an image obtained from the Internet (i.e., it is very likely not an image taken by the user through the user terminal 10). The third determination unit 24 uses template matching to compare the image with Internet images sent to the backend (parts invisible to the user's eye, such as the server side or database system), and measures the similarity between the two. The third determination unit 24 can also calculate the probability (reliability score) that the image input to the image input unit 21 is an existing image lacking originality.

[0064] The angle calculation unit 25 analyzes the image input to the image input unit 21, determines a first straight line and a second straight line based on the contour features of the beverage can, and calculates the angle formed by the first straight line and the second straight line. In this disclosure, the center point of the upper surface of the beverage can, the center point and tip of the drinking spout on the upper surface of the beverage can, and the reference point of the marking portion on the side of the beverage can are used. Furthermore, the contour features of the beverage can include at least one of the drinking spout on the upper surface of the beverage can, the center portion of the upper surface of the beverage can, and the display portion on the side of the beverage can. In this specification, "…point" and "…part" are interchangeable. For example, the center point of the upper surface of the beverage can, the center point and tip of the drinking spout on the upper surface of the beverage can, and the reference point of the marking portion on the side of the beverage can can be replaced with the center portion of the upper surface of the beverage can, the center portion and tip of the drinking spout on the upper surface of the beverage can, and the reference portion of the marking portion on the side of the beverage can.

[0065] Figure 6 This diagram illustrates an example of the structure of the angle calculation unit 25. The angle calculation unit includes a first line acquisition unit 25X, a second line acquisition unit 25Y, and an angle acquisition unit 25Z.

[0066] The first line acquisition unit 25X acquires a first line from the image input to the image input unit 21. This first line connects the center point of the upper surface of the beverage can and the center point of the drinking spout of the beverage can, or the tip of the drinking spout that is farthest from the center point of the upper surface. The first line acquisition unit 25X performs a first acquisition process of acquiring the first line connecting the center point of the upper surface of the beverage can to the tip of the drinking spout that is farthest from the center point of the upper surface. If the first acquisition process fails, it performs a second acquisition process of acquiring the first line connecting the center point of the upper surface of the beverage can to the center point of the drinking spout of the beverage can. For example, if the drinking spout of the beverage can is elliptical or approximately elliptical, the intersection of the major axis and the minor axis, i.e., the center of the ellipse or approximately ellipse, corresponds to the "center point of the drinking spout of the beverage can". Alternatively, the first line may also be defined as a line connecting the tip (point) of the drinking spout on the upper surface of the beverage can to the center (point) of the upper surface of the beverage can. Alternatively, the first straight line can also be defined as the straight line connecting the farthest point (part) of the drinking spout on the upper surface of the beverage can to the center point (part) of the upper surface of the beverage can.

[0067] The second line acquisition unit 25Y acquires a second line from the image input to the image input unit 21. This second line connects the center point of the upper surface of the beverage can and the point where a line extending vertically from the reference point of the label on the side of the beverage can toward the plane containing the upper surface intersects the plane containing the upper surface. The "reference point of the label on the side of the beverage can" is defined as a specific part of the manufacturer or product name of the beverage can indicated by the label. For example, if the label on the side of the beverage can is drawn as XYZ as the manufacturer of the beverage can and ABCDE as the product name of the beverage can, a specific part of XYZ (e.g., the center of the letter Y) or a specific part of ABCDE (e.g., the center of the letter C) is set as the "reference point of the label on the side of the beverage can". Details will be described later. The "reference point of the label on the side of the beverage can" is extracted and determined using segmentation and key point extraction (especially key point extraction), targeting a point of the label on the side of the beverage can (e.g., the center of the letter Y in the letter XYZ, or the center of the letter C in the letter ABCDE). Alternatively, the second straight line can be defined as a straight line connecting the point where it intersects the upper surface of the beverage can (including a point on the side of the display portion) and the center point of the upper surface of the beverage can. Alternatively, the second straight line can be defined as a straight line connecting the point where it intersects the upper surface of the beverage can (including a point on the side of the display portion) and the center point (part) of the upper surface of the beverage can.

[0068] As described above, the top surface of a beverage can is circular (approximately circular) when viewed from above. Therefore, when taking an image of a beverage can, if the image is taken from directly above the can, the top surface appears circular (approximately circular). However, if the image is taken at a slight angle, the top surface appears non-circular, and it is very likely that the latter is the case in almost all cases. Therefore, the first straight line acquisition unit 25X acquires a first straight line based on correcting the non-circular top surface of the beverage can in the image to a perfect circle, and the second straight line acquisition unit 25Y acquires a second straight line based on correcting the non-circular top surface of the beverage can in the image to a perfect circle.

[0069] The angle acquisition unit 25Z acquires the angle formed by the first straight line acquired by the first straight line acquisition unit 25X and the second straight line acquired by the second straight line acquisition unit 25Y.

[0070] Figure 7 This diagram illustrates the configuration of a first and second straight line, and an example of the angle between them. Figure 7In this diagram, the reference point of the markings on the side of the beverage can, specifically the center of the 'C' in the product name ABCDE and / or the center of the 'Y' in the manufacturer's name XYZ, is drawn as point 1. Furthermore, the point where a straight line extending perpendicularly from reference point 1 on the side of the beverage can towards the plane containing the upper surface of the beverage can intersects the plane is drawn as point 2. Additionally, the center point of the upper surface of the beverage can is drawn as point 3. Furthermore, the tip of the drinking spout furthest from the center point of the upper surface is drawn as point 4. Finally, the center point of the drinking spout is drawn as point 5.

[0071] Then, a first straight line is obtained connecting the center point 3 of the upper surface of the beverage can and the center point 5 of the drinking spout of the beverage can, or the tip 4 of the drinking spout that is farthest from the center point of the upper surface. More specifically, as a first acquisition process, the acquisition process of the first straight line connecting the center point 3 of the upper surface of the beverage can and the tip 4 of the drinking spout that is farthest from the center point of the upper surface is executed. If the first acquisition process fails, as a second acquisition process, the acquisition process of the first straight line connecting the center point 3 of the upper surface of the beverage can and the center point 5 of the drinking spout of the beverage can is executed. On the other hand, a second straight line is obtained, connecting the center point 3 of the upper surface of the beverage can and the point 2 where a straight line extending perpendicularly from the reference point 1 of the marking part on the side of the beverage can intersects the plane containing the upper surface. The angle θ formed by the first straight line and the second straight line obtained in this way is obtained. In addition, when the first straight line and the second straight line are obtained, if the upper surface of the beverage can is not circular (elliptical), the calculated angle can be corrected. Furthermore, as described above, the first straight line and the second straight line can also be obtained by correcting the upper surface of the beverage can projected into the image from a non-circular shape to a true circle.

[0072] Angle calculations based on angle calculation units 25 (first line acquisition unit 25X, second line acquisition unit 25Y, and angle acquisition unit 25Z) are performed through three processes: segmentation, key point extraction, and angle calculation based on geometry processing. Specifically, when an image containing a beverage can is sent to a backend (a part of the system that the user's eyes cannot see, such as a server or database), these three processes—segmentation, key point extraction, and angle calculation based on geometry processing—are performed. The result is that a first line and a second line are determined, and the angle formed by the first line and the second line is obtained. Thus, the first line acquisition unit 25X and the second line acquisition unit 25Y obtain the first and second lines based on segmentation and key point extraction.

[0073] Segmentation is a model that infers the content displayed for each pixel in an image. It can be used to independently create teaching data and perform supplementary learning, employing model architectures typically used in object detection. In segmentation, pixels are used to determine the location of the upper surface region of a beverage can in the target image.

[0074] Keypoint extraction is a model for detecting the coordinates of points that become features of an object in an image. It can be used to define new keypoints for a beverage can in an image and learn from the model. In keypoint extraction, the coordinates of points 1 to 5 are detected to define (specifically specify) the first and second straight lines in the target image.

[0075] In the angle calculation of geometric processing, the angle θ formed by the first straight line (the line connecting points 3 and 4 or the line connecting points 3 and 5) and the second straight line (the line connecting points 2 and 3) is calculated based on the data detected through segmentation and keypoint extraction. Figure 7 ).

[0076] As an example, in segmentation, the image projected onto the beverage can is used as input to determine the location of the upper region of the can in pixels. In keypoint extraction, the image projected onto the beverage can is used as input to specifically designate the center point of the upper surface of the can, the reference point of the marker, the tip of the drinking spout, or the center point. In the angle calculation for geometric processing, the results of segmentation and keypoint extraction are used as input to calculate a first line and a second line, and obtain the angle formed by the first line and the second line.

[0077] When multiple images from the same user terminal 10 are input into the image input unit 21, the angle calculation unit 25 (first line acquisition unit 25X, second line acquisition unit 25Y, angle acquisition unit 25Z) calculates the angle θ formed by the first line (the line connecting points 3 and 4 or the line connecting points 3 and 5) and the second line (the line connecting points 2 and 3) for the multiple images.

[0078] The angle output unit 26 outputs the angle θ formed by the first straight line (the line connecting points 3 and 4 or the line connecting points 3 and 5) and the second straight line (the line connecting points 2 and 3), calculated by the angle calculation unit 25 (first straight line acquisition unit 25X, second straight line acquisition unit 25Y, and angle acquisition unit 25Z). When multiple images from the same user terminal 10 are input to the image input unit 21, and the angle calculation unit 25 calculates the angle θ formed by the first and second straight lines for these multiple images, the angle output unit 26 outputs the angle θ formed by the first and second straight lines for each of the multiple images.

[0079] The comparison unit 27 compares multiple images from the same user terminal 10 to see if the angle θ formed by the first straight line and the second straight line is within a specified range.

[0080] When manufacturing beverage cans, the positional relationship between the drinking spout on the top surface and the markings (manufacturer, product name) on the side of the can is not always identical. This positional relationship can be considered a measure of the originality of each beverage can. Strictly speaking, for each beverage can, almost every single one has a shell with a different angle θ formed by a first straight line and a second straight line. This angle θ can be used to determine the likelihood of the same beverage can appearing in multiple images. For example, if the angle θ formed by the first and second straight lines in multiple images is significantly different, the likelihood of the beverage cans appearing in the multiple images is high; conversely, if the angle θ formed by the first and second straight lines is the same or nearly the same, the likelihood of the beverage cans appearing in the multiple images is high.

[0081] The comparison unit 27, for example, sets the angle θ formed by the first straight line and the second straight line in any one of the multiple reference images as the reference angle θ. standard Furthermore, the angle θ formed by the first and second straight lines in other reference images of multiple images is set as the reference angle θ. reference Then compare with the reference angle θ. standard With reference angle θ reference Whether it is within the specified range. For example, satisfying 0.95 × reference angle θ. reference ≤Reference angle θ standard ≤1.05 × reference angle θ reference If the condition is met, it can be determined that the reference image contains the same beverage can as the reference image; if the condition is not met, it can be determined that the reference image contains different beverage cans from the reference image. The comparison unit 27 can also calculate the probability (reliability score) indicating whether the beverage cans contained in multiple images (reference image and reference image) are the same.

[0082] When multiple images from the same user terminal 10 are within a predetermined range at an angle θ formed by the first and second straight lines (i.e., when there is a high probability that the beverage cans contained in the multiple images are the same), the notification unit 28 notifies the user terminal 10 of the above content. The notification method to the user terminal 10 may include, for example, displaying on a screen and / or sound from a speaker. In this case, the notification unit 28 may notify that the multiple images are not eligible for the benefits granted by the benefit granting unit 29, which will be described later.

[0083] Furthermore, when the area of ​​the beverage can in the image containing the beverage can input to the image input unit 21 by the notification unit 28 is below a specified range, the angle θ formed by the first straight line and the second straight line cannot be calculated (obtained), and an error notification may be issued.

[0084] For multiple images from the same user terminal 10, if the angle θ formed by the first and second straight lines is not within a specified range, the discount assignment unit 29 determines that the beverage cans contained in the multiple images are different, and assigns a discount to the user terminal 10 corresponding to the number of images. For example, one image containing a beverage can can be converted into 1 point, and when 5 points are reached corresponding to 5 images, an automatic application is made, and original merchandise can be won in the event of a lottery (see reference). Figure 4B , Figure 4C ).

[0085] Figure 8 This is a diagram illustrating an example of the concept of an application service based on images containing beverage cans. Figure 8 As shown, an image containing a beverage can is sent from the application on user terminal 10 to the AI ​​(Artificial Intelligence) of image processing device 20. The AI ​​of image processing device 20 assigns information about the beverage can to the uploaded image. More specifically, it can obtain information such as the angle of the drinking spout of the beverage can (e.g., the angle θ formed by the first and second straight lines), the similarity to images on the Internet, the angle of the drinking spout of beverage cans recently uploaded by the same user (e.g., the angle θ formed by the first and second straight lines), and product information (e.g., whether the beverage can is 350ml or 500ml). The result determined by the AI ​​of image processing device 20 is sent to the activity system in numerical form and is configured to allow for adjustments to award points and discounts on the activity system side.

[0086] Figure 9 This is a diagram illustrating an example of a process for processing an image containing a beverage can.

[0087] In step ST1, the camera of the camera unit 11, which is the user terminal 10, is activated. In step ST2, the camera unit 11 of the user terminal 10 captures (acquires) an image containing a beverage can. If the logo or a specific part of the beverage can (e.g., text or pattern indicating the product name) is recognized while the camera is activated (if the probability of the target beverage can exceeds a predetermined value), the shutter is automatically pressed to capture an image containing the beverage can.

[0088] In step ST3, the second determination unit 23 of the image processing device 20 determines that the beverage can is of the specified capacity. This determination can be made, for example, by distinguishing between 500ml and 350ml based on the overall shape of the beverage can, or by using the product ID (JAN code) attached to the beverage can. In step ST4, the third determination unit 24 of the image processing device 20 confirms that the image containing the beverage can is not an existing image lacking originality (an image obtained from the Internet).

[0089] In step ST5, the angle calculation unit 25 (first line acquisition unit 25X, second line acquisition unit 25Y, and angle acquisition unit 25Z) of the image processing device 20 calculates the angle formed by the first line and the second line for the beverage cans included in the captured image. In step ST6, the comparison unit 27 of the image processing device 20 checks against the database, comparing the angle formed by the first line and the second line for beverage cans included in the most recently registered image of a user with the same ID (same user) with the angle formed by the first line and the second line calculated in step ST5, and determines their consistency. In step ST7, it is finally determined whether the beverage cans included in the multiple images for which angle comparison and determination were performed in step ST6 are the same beverage can or different beverage cans.

[0090] Thus, in the image processing apparatus containing a beverage can disclosed herein, a first straight line acquisition unit acquires a first straight line from the image containing the beverage can. This first straight line connects the center point of the upper surface and the center point of the drinking spout, or the tip of the drinking spout furthest from the center point of the upper surface. Furthermore, a second straight line acquisition unit acquires a second straight line from the image containing the beverage can. This second straight line connects the center point of the upper surface and the point where a straight line extending perpendicularly from a reference point of the side marking portion towards the plane containing the upper surface intersects the plane. Additionally, an angle acquisition unit acquires the angle formed by the first and second straight lines. This allows for suitable (improved) image processing of the beverage can. For example, it allows for the analysis of beverage cans included in the image to confirm whether the beverage cans included in the image are identical, thereby obtaining proof of purchase for the beverage can. Furthermore, the present invention is flexible and convenient, and can achieve a certain level of accuracy.

[0091] Furthermore, in the image processing apparatus containing beverage cans disclosed herein, flexibility in judging correctness is ensured by storing probabilities as whether the image contains a specific beverage can or the same beverage can. Moreover, by providing detailed feedback on relevant metadata, flexible changes can be made in activity design. Furthermore, since parsing processing is performed on the application server side (image processing apparatus 20) and beverage can recognition is performed on the application side (user terminal 10), a real-time user experience (UX) can be achieved. Furthermore, processing can be implemented, for example, using JavaScript (registered trademark), which runs in HTML, making installation easy.

[0092] Figure 10 This diagram illustrates an example of the functional structure of a user terminal 10 and an image processing apparatus 20 according to one embodiment. For example... Figure 10 As shown, the user terminal 10 and the image processing device 20 include a control unit 110, a storage unit 120, a communication unit 130, an input unit 140, and an output unit 150. Furthermore, in Figure 10 An example of a functional block diagram showing the user terminal 10 and the image processing device 20 is shown, which may also have Figure 10 Other function boxes not drawn in the diagram. Alternatively, it can be a structure that does not include any function boxes.

[0093] The control unit 110 implements control over the user terminal 10 and the image processing device 20. The control unit 110 may be composed of a controller, control circuit, or control device described based on consensus in the technical field to which this invention pertains.

[0094] The storage unit 120 stores information used in the user terminal 10 and the image processing device 20. The storage unit 120 may be configured, for example, by a memory, storage device, or the like, based on consensus in the technical field to which this invention pertains.

[0095] The communication unit 130 performs communication between the user terminal 10 and the image processing device 20 (e.g., mutual communication between the two). The communication unit 130 may be composed of a transmitter and receiver, a transceiver circuit, or a transceiver device, as described based on consensus in the technical field to which this invention pertains. Alternatively, the communication unit 130 may also be composed of a transmitting unit and a receiving unit.

[0096] The input unit 140 receives input from the user terminal 10 and the image processing device 20. Furthermore, the input unit 140 can also be connected to a specified device, storage medium, etc., to receive data input. The input unit 140 can also output input results to, for example, the control unit 110. The input unit 140 can be configured with input devices such as keyboards, mice, and buttons, input and output terminals, input and output circuits, etc., as described based on consensus in the technical field to which this invention pertains. Furthermore, the input unit 140 can also be an integrated structure with the display unit (e.g., a touch panel).

[0097] The output unit 150 outputs data to the user terminal 10 and the image processing device 20. For example, the output unit 150 may be configured to include a display unit for displaying images, a sound output unit for outputting sound, etc. The display unit may be, for example, a display device such as a monitor or display screen, as described based on consensus in the technical field to which this invention pertains. Furthermore, the sound output unit may be configured to be an output device such as a speaker, as described based on consensus in the technical field to which this invention pertains.

[0098] Furthermore, the block diagrams used in the above description of the embodiments represent functional units. These functional blocks (structural parts) are implemented through any combination of hardware and / or software. Moreover, there are no particular limitations on the implementation method of each functional block. That is, each functional block can be implemented by a physically combined device, or by connecting two or more physically separate devices via wired or wireless connections, and implemented by these multiple devices.

[0099] For example, the user terminal 10 and the image processing device 20 in one embodiment of this disclosure can also function as a computer for processing the image processing method and processing program of the beverage can disclosed herein. Figure 11 This diagram illustrates an example of the hardware structure of a user terminal 10 and an image processing apparatus 20 according to one embodiment. The user terminal 10 and image processing apparatus 20 described above can also be physically configured as a computer device including a processor 1001, a memory 1002, a storage device 1003, a communication device 1004, an input device 1005, an output device 1006, a bus 1007, etc.

[0100] Furthermore, in this disclosure, devices, circuits, equipment, units, servers, etc., are interchangeable. The hardware structure of the user terminal 10 and the image processing device 20 can be configured to include one or more of the devices shown in the figures, or it can be configured to exclude any of them.

[0101] For example, only one processor 1001 is illustrated, but there can be multiple processors. Furthermore, processing can be executed by one processor, or processing can be executed simultaneously, sequentially, or in other ways by two or more processors. Additionally, processor 1001 can be implemented using more than one chip.

[0102] The functions in the user terminal 10 and the image processing device 20 are implemented by loading prescribed software (programs) onto the hardware such as the processor 1001 and the memory 1002, enabling the processor 1001 to perform calculations and control communication based on the communication device 1004, reading and / or writing of data in the memory 1002 and the storage device 1003.

[0103] The processor 1001, for example, enables the operating system to run in order to control the computer as a whole. The processor 1001 may be composed of a central processing unit (CPU) that includes interfaces with peripheral devices, control devices, arithmetic units, registers, etc. Furthermore, the various components such as the control unit 110 mentioned above can be implemented by the processor 1001.

[0104] Furthermore, the processor 1001 reads programs (program code), software modules, data, etc., from at least one of the memory 1003 and the communication device 1004 into the memory 1002, and performs various processes accordingly. The program is a program that causes the computer to perform at least a portion of the operations described in the above embodiments. For example, the control unit 110 can be implemented by a control program stored in the memory 1002 and running in the processor 1001, and other functional blocks can also be implemented similarly.

[0105] Memory 1002 is a computer-readable storage medium, and may be composed of at least one of ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically EPROM) (registered trademark), RAM (Random Access Memory), or other suitable storage media. Memory 1002 may be referred to as a register, cache, main memory (main storage device), etc. Memory 1002 may store programs (program code), software modules, etc., that are executable for implementing the methods involved in one embodiment.

[0106] The memory 1003 is a computer-readable storage medium, and may be constituted by at least one of the following: a flexible disk, a floppy disk (registered trademark), a magneto-optical disk (e.g., an optical disc (CD-ROM (Compact Disc ROM)), a digital multipurpose disk, a Blu-ray disc (registered trademark), a removable disk, a hard disk drive, a smart card, a flash memory device (e.g., a card, a stick, a key drive), a magnetic stripe, a database, a server, or other suitable storage media. The memory 1003 may be referred to as an auxiliary storage device. Furthermore, the aforementioned storage unit 120 may be implemented using the main memory 1002 and / or the memory 1003.

[0107] The communication device 1004 is hardware (transmitting and receiving device) used for communication between computers via at least one of a wired network and a wireless network. It is also referred to as a network device, network controller, network interface card (NIC), communication module, etc. The communication device 1004 may also include a SIM card. Furthermore, the aforementioned communication unit 130 can be implemented through the communication device 1004.

[0108] Input device 1005 is an input device (e.g., keyboard, mouse, etc.) that receives input from external sources. Output device 1006 is an output device (e.g., display, speaker, etc.) that performs output to external sources. Furthermore, input device 1005 and output device 1006 can be integrated into a single structure (e.g., a touch panel). Additionally, the aforementioned input section 140 and output section 150 can be implemented using input device 1005 and output device 1006, respectively.

[0109] Furthermore, the processor 1001, memory 1002, and other devices are connected via a bus 1008 for communication information. The bus 1007 can be a single bus or can be composed of different buses between devices.

[0110] Furthermore, the user terminal 10 and the image processing device 20 can be configured as hardware including a microprocessor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), an FPGA (Field Programmable Gate Array), etc., and can also implement part or all of the functional blocks through such hardware. For example, the processor 1001 can be implemented using at least one of these hardware components.

[0111] (Modified Example)

[0112] Furthermore, the terms used in this disclosure and / or those required for understanding this disclosure may be replaced with terms having the same or similar meanings.

[0113] Information, parameters, etc., described in this disclosure may be expressed in absolute values, relative values ​​of specified values, or other corresponding information. Furthermore, the names used for parameters, etc., in this disclosure are not limited in any way.

[0114] The information, signals, etc. described in this disclosure can be represented by any of various technologies. For example, data, commands, instructions, information, signals, bits, symbols, chips, etc. mentioned in the foregoing description can be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, light fields or photons, or any combination thereof.

[0115] Information and signals can be input / output via multiple network nodes. The input / output information and signals can be stored in a specific location (e.g., memory) or managed using tables. The input / output information and signals can be overwritten, updated, or added. The output information and signals can be deleted. The input information and signals can be sent to other devices.

[0116] Furthermore, notification of prescribed information (such as a notification of "is X") is not limited to explicit notification, but can also be implicit (e.g., by not giving notification of the prescribed information or by giving notification of other information).

[0117] Whether it is called software, firmware, middleware, microcode, hardware description language, or any other name, software should be broadly interpreted as meaning instructions, instruction sets, code, code segments, program code, programs, subroutines, software modules, application programs, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, etc.

[0118] Furthermore, software, commands, information, etc., can be sent and received via at least one of a transmission medium and a signal waveform. For example, when software is sent from a website, server, or other remote source using at least one of wired technologies (coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), etc.) and wireless technologies (infrared, microwave, etc.), at least one of these wired and wireless technologies is included within the definition of a transmission medium.

[0119] The terms “system” and “network” used in this disclosure are used interchangeably.

[0120] The various methods and implementations described in this disclosure can be used individually, in combination, or switched during execution. Furthermore, as long as there is no contradiction, the processing procedures, sequences, processes, etc., of the various methods and implementations described in this disclosure can be changed in order. For example, regarding the methods described in this disclosure, the elements of each step are indicated by an illustrated order, but are not limited to the specific order indicated.

[0121] As used in this disclosure, the term "based on" does not mean "based on only" unless otherwise specified. In other words, the term "based on" implies both "based on only" and "based on at least".

[0122] The arbitrary use of designations such as "first," "second," etc., as used in this disclosure does not generally limit the quantity or order of these elements. These designations can be used as a convenient method to distinguish between two or more elements in this invention. Therefore, the reference to a first element and a second element does not imply that only two elements can be used or that the first element must in some form precede the second element.

[0123] In this disclosure, the terms “comprising,” “including,” and variations thereof are intended to have the same comprehensive meaning as the term “possessing.” Furthermore, the term “or” as used in this disclosure does not mean XOR.

[0124] In this disclosure, for example, when articles are added by translation for a, an, and the in English, this disclosure may include nouns that are plural after these articles.

[0125] The inventions disclosed herein have been described in detail above. However, it will be apparent to those skilled in the art that the inventions are not limited to the embodiments described herein. The inventions disclosed herein can be implemented with modifications and variations without departing from the spirit and scope of the invention as defined by the claims. Therefore, the purpose of this disclosure is illustrative and does not impose any limiting effect on the inventions disclosed herein.

[0126] This application is based on Japanese Special Petition 2023-026013, filed on February 22, 2023. Its entire contents are contained herein.

Claims

1. A processing apparatus comprising an image of a beverage can, the beverage can having: The upper surface, when viewed from above, is roughly circular; The lower surface, when viewed from above, is roughly circular; From the side, it is cylindrical, and from top view it is roughly ring-shaped and connects the upper surface and the lower surface; The drinking spout is located between the center point and the outer periphery of the upper surface; and The marking section is located on the side. wherein The processing apparatus containing the image of the beverage can is characterized by having: An image acquisition unit acquires an image containing the beverage can, captured by the camera unit of the user terminal and transmitted by the image transmission unit of the user terminal; The first straight line acquisition unit acquires a first straight line from the image, the first straight line connecting the center point of the upper surface and the center point of the drinking spout or the tip of the drinking spout that is farthest from the center point of the upper surface. The second straight line acquisition unit acquires a second straight line from the image, the second straight line connecting the center point of the upper surface and the point where a straight line extending perpendicularly from the reference point of the marking portion on the side surface toward the plane containing the upper surface intersects the plane; and Angle acquisition unit acquires the angle formed by the first straight line and the second straight line.

2. The processing apparatus for an image containing a beverage can according to claim 1, characterized in that, The first straight line acquisition unit and the second straight line acquisition unit acquire the first straight line and the second straight line based on correcting the upper surface of the beverage can projected into the image from a non-circular shape to a perfect circle.

3. The processing apparatus for an image containing a beverage can according to claim 1 or 2, characterized in that, The first straight line acquisition unit performs a first acquisition process of acquiring a first straight line connecting the center point of the upper surface to the tip of the drinking mouth that is furthest from the center point of the upper surface. If the first acquisition process fails, it performs a second acquisition process of acquiring a first straight line connecting the center point of the upper surface to the center point of the drinking mouth.

4. The processing apparatus for an image containing a beverage can according to claim 1 or 2, characterized in that, The first line acquisition unit and the second line acquisition unit acquire the first line and the second line based on segmentation and key point extraction.

5. A method for processing an image containing a beverage can, the beverage can having: The upper surface, when viewed from above, is roughly circular; The lower surface, when viewed from above, is roughly circular; From the side, it is cylindrical, and from top view it is roughly ring-shaped and connects the upper surface and the lower surface; The drinking spout is located between the center point and the outer periphery of the upper surface; and The marking section is located on the side. in, The method for processing images containing beverage cans is characterized by the following steps being performed by a computer: The step of acquiring an image containing the beverage can by means of an image acquisition unit, which is captured by the camera unit of the user terminal and transmitted by the image transmission unit of the user terminal; The step of obtaining a first straight line from the image by a first straight line obtaining unit, wherein the first straight line connects the center point of the upper surface and the center point of the drinking spout or the tip of the drinking spout that is farthest from the center point of the upper surface; The step of obtaining a second straight line from the image via a second straight line acquisition unit, wherein the second straight line connects the center point of the upper surface and the point where a straight line extending perpendicularly from the reference point of the marking unit on the side surface toward the plane containing the upper surface intersects the plane; and The step of obtaining the angle formed by the first straight line and the second straight line through the angle acquisition unit.

Images