Processing device, processing method, and processing program for images including containers

Abstract

To provide a processing device, a processing method, and a processing program of an image including a suitable (improved) container.SOLUTION: A processing device of an image including a container comprises an upper surface open part, a lower surface bottom part, a side surface connection part connecting the upper surface open part and the lower surface bottom part, an upper surface sealing part sealing the upper surface open part, an upper surface side display part provided in the upper surface sealing part, and a side surface side display part provided in the side surface connection part. The processing device of the image including the container comprises: an image acquisition unit which acquires the image including the container, captured by a capturing unit of a user terminal and transmitted by an image transmission unit of the user terminal; a first straight line acquisition unit which acquires a first straight line based on the upper surface side display part, from the image; a second straight line acquisition unit which acquires a second straight line based on the side surface side display part, from the image; and an angle acquisition unit which acquires an angle defined by the first straight line and the second straight line.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present disclosure relates to a processing device, a processing method, and a processing program for an image including a container. [Background technology]

[0002] Patent Document 1 describes a drunk driving prevention device that includes an individual identification information reading means, a position estimation means, and a drunk driving prevention control means. The individual identification information reading means is capable of reading individual identification information from an alcoholic beverage container present in the vehicle cabin. The position estimation means estimates the position of the alcoholic beverage within the vehicle cabin based on the output of the individual identification information reading means. The drunk driving prevention control means performs predetermined control to prevent drunk driving based on the position of the alcoholic beverage estimated by the position estimation means. Patent Document 1 also exemplifies an RFID reader as a specific example of a device that reads the individual identification information of an alcoholic beverage. Patent Document 1 further describes that the device that reads the individual identification information of an alcoholic beverage is not limited to one that uses wireless communication, and may also be one that captures an image of a barcode or the like and reads the individual identification information of the alcoholic beverage by image recognition, for example.

[0003] Patent Document 2 describes a notification control system that includes a user information receiving means, a store information receiving means, a matching means, and a purchase information notification means. The user information receiving means receives user information about beverages posted by users. The store information receiving means receives store information about beverage purchases at physical or virtual stores. The matching means matches the user information with store information regarding beverage attributes. The purchase information notification means notifies the user of purchase information based on the user information and store information matched by the matching means. Patent Document 1 also describes that the matching means performs image recognition on image data of the user information to identify beverage attributes, and that publicly known image recognition technology (OCR technology, shape determination technology, etc.) may be used to perform image recognition on the image data of the user information to identify beverage attributes. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-258855 [Patent Document 2] Japanese Patent Application Publication No. 2017-091343 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the prior art including Patent Documents 1 and 2 leaves room for improvement in terms of processing images including suitable (improved) containers.

[0006] Therefore, one of the objects of the present disclosure is to provide a processing device, a processing method, and a processing program for an image including a suitable (improved) container. [Means for solving the problem]

[0007] The image processing device including a container according to one aspect of the present disclosure includes an upper surface opening portion, a lower surface bottom portion, a side connection portion connecting the upper surface opening portion and the lower surface bottom portion, an upper surface sealing portion sealing the upper surface opening portion, and Consists of a string The upper display unit and the side connection unit are provided Consists of a reference point and a side display unit, the image processing device including the container, the image processing device including the container having the side display unit, the image processing device including the container having the side display unit and ... the first straight line acquisition unit acquires, as the first straight line, a straight line along the character string or a straight line along the boundary line of the adjacent character string, and acquires, as an updated version of the first straight line, a straight line along the character string or a straight line along the boundary line of the adjacent character string, translated so that the straight line passes through a center point of the top sealing unit; and the second straight line acquisition unit acquires, as the second straight line, a straight line connecting the center point of the top sealing unit and a point at which a straight line extending perpendicularly from the reference point of the side display unit toward a plane including the top sealing unit intersects with the plane including the top sealing unit. It is characterized by: An image processing device for an image including a container according to one aspect of the present disclosure is a processing device for an image including a container having a top opening portion, a bottom portion, a side connection portion connecting the top opening portion and the bottom portion, a top sealing portion sealing the top opening portion, a top display portion including a character string provided on the top sealing portion, and a side display portion including a reference point provided on the side connection portion, and the device has an image acquisition unit that acquires an image including the container captured by an imaging unit of a user terminal and transmitted by an image transmission unit of the user terminal, a first line acquisition unit that acquires a first line based on the top display portion from the image, a second line acquisition unit that acquires a second line based on the side display portion from the image, and an angle acquisition unit that acquires an angle between the first line and the second line, and the first line acquisition unit acquires a line along the character string or an adjacent line A straight line along the boundary line of the character string is acquired as the first straight line, and if the top surface sealing portion reflected in the image is a perfect circle, a straight line along the character string or a straight line along the boundary line of the adjacent character string is translated parallel to pass through the center point of the perfect circle and acquired as an updated version of the first straight line, and if the top surface sealing portion reflected in the image is an ellipse, a straight line along the character string or a straight line along the boundary line of the adjacent character string is translated parallel to pass through the center point of the major axis or minor axis of the ellipse and acquired as an updated version of the first straight line, and the second straight line acquisition unit acquires as the second straight line a straight line connecting the center point of the top surface sealing portion and a point where a straight line extending perpendicularly from the reference point of the side display unit toward a plane including the top surface sealing portion intersects with the plane including the top surface sealing portion. [Effects of the Invention]

[0008] According to one aspect of the present disclosure, it is possible to obtain a processing device, a processing method, and a processing program for an image including a suitable (improved) container. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a system including a processing device for an image including a container. [Figure 2] FIG. 2 is a diagram illustrating an example of the configuration of a user terminal. [Figure 3] FIG. 10 is a diagram showing an example of an image including a container. [Figure 4] FIG. 10 is a diagram illustrating an example of the concept of a container image analysis application service based on an image including a container. [Figure 5] FIG. 1 illustrates an example of a configuration of an image processing apparatus. [Figure 6] FIG. 2 illustrates an example of the configuration of an angle calculation unit. [Figure 7] 10A and 10B are diagrams illustrating an example of a first line acquisition process performed by a first line acquisition unit. [Figure 8] 10 is a diagram showing an example of a process of acquiring a second line by a second line acquisition unit; FIG. [Figure 9] 4 is a diagram showing an angle formed between a first line and a second line acquired by an angle acquisition unit. FIG. [Figure 10] FIG. 10 is a diagram illustrating an example of the concept of a container image analysis application service based on an image including a container. [Figure 11] FIG. 10 is a diagram showing a first example of the flow of a method for processing an image including a container. [Figure 12] FIG. 10 is a diagram showing a second example of the flow of a method for processing an image including a container. [Figure 13] FIG. 10 is a diagram illustrating an example of a functional configuration of a user terminal and a processing device for an image including a container. [Figure 14] FIG. 10 is a diagram illustrating an example of a hardware configuration of a user terminal and a processing device for an image including a container. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0011] FIG. 1 is a diagram showing an example of the configuration of a system 1 including a processing device 20 for images including containers. The "processing device for images including containers" may be read as a "processing system for images including containers" (these may be synonymous). The method and program for processing images including containers are realized by causing a computer (CPU: Central Processing Unit) included in the system 1 (user terminal 10, processing device 20 for images including containers) to execute various processing steps.

[0012] As shown in FIG. 1, the system 1 includes a user terminal 10, a processing device for images including containers (hereinafter referred to as the "image processing device") 20, and a network 30 connecting the user terminal 10 and the image processing device 20. In FIG. 1, the user terminal 10 and the image processing device 20 are depicted as a simplified single configuration for the sake of convenience of drawing. In reality, the user terminal 10 and the image processing device 20 have more complex configurations than those depicted in FIG. 1, both in terms of hardware and software configuration. For example, the image processing device 20 may be configured to include multiple servers, databases, etc.

[0013] The system 1 may be understood as a configuration including a user terminal 10 and an image processing device 20, or as a configuration including only the image processing device 20 without the user terminal 10. In other words, the user terminal 10 is not an essential component of the system 1 (it is sufficient that an image including a container, which will be described later, is input to the image processing device 20).

[0014] The user terminal 10 is, for example, a mobile terminal (mobile communication terminal) such as a mobile phone, a smartphone, or a tablet terminal. In other words, the user terminal 10 in this specification is a communication device capable of performing various types of communication with the image processing device 20. The user terminal 10 may be, for example, a wired and / or wireless (e.g., Long Term Evolution (LTE), thThe user terminal 10 may communicate with a network (such as the Internet) via a 5G Generation New Radio (5G NR), Wi-Fi (registered trademark), etc. The user terminal 10 executes a browser and various other applications (particularly a container image analysis application service described later) in response to a user operation.

[0015] The image processing device 20 may be any communication device capable of performing various types of communication with the user terminal 10, and may be interpreted as a device, circuit, etc. In particular, the image processing device 20 performs various types of communication with the user terminal 10 to analyze an image including a container acquired (captured) by the user terminal 10 and provide various application services (container image analysis application services) based on the analysis results.

[0016] Fig. 2 is a diagram showing an example of the configuration of the user terminal 10. The user terminal 10 has an imaging unit (image acquisition unit) 11, an image transmission unit 12, and a container image analysis application unit 13. In addition to the components shown in Fig. 2, the user terminal 10 also has components that are installed in a general user terminal, such as a display, a speaker, and a microphone.

[0017] The imaging unit 11 is a camera system mounted on the user terminal 10, and captures (obtains) an image including a container (a container before opening). The image transmission unit 12 transmits the image including the container captured by the imaging unit 11 to the image processing device 20. The container image analysis application unit 13 executes various processes for utilizing a container image analysis application service based on the image including the container transmitted from the image transmission unit 12 to the image processing device 20.

[0018] For example, when the camera of the imaging unit 11 is activated, a photograph may be taken by determining whether the container in the image received by the imaging element (CCD: Charge Coupled Device) is a predetermined brand. To achieve a more comfortable user experience (UX) than capturing an image separately, the container determination process may be separated. In the determination process, for example, the outer shape and design of the container may be analyzed to determine whether the image received by the imaging element includes the logo of the predetermined brand or the top surface of the container. In this case, the probability (reliability score) that the container imaged by the imaging element is a predetermined brand may be calculated, and imaging may be performed if the probability is equal to or greater than a predetermined value (e.g., 50% or greater). The predetermined value that determines whether or not imaging is performed has a degree of freedom, allowing for various design modifications. Of course, whether the container imaged by the imaging element is a predetermined brand may also be detected using a 1 / 0 flag.

[0019] 3A, 3B, and 3C are diagrams showing examples of images including a container. As shown in FIGS. 3A to 3C, a container 40 has a top opening 41, a bottom portion 42, a side connection portion 43 connecting the top opening 41 and the bottom portion 42, and a top sealing portion 44 sealing the top opening 41. FIGS. 3A to 3C show the state of the container 40 before opening, in which the top opening 41 is sealed by the top sealing portion 44, and the top opening 41 is hidden by the top sealing portion 44 (the leader line of the top opening 41 is drawn with a dashed line). The image processing of this embodiment targets an image including the container 40 before opening. In addition, in this specification, the "shape of the top sealing portion 44 (a perfect circle or ellipse, as described below)" refers to the top surface of the top sealing portion 44 (a surface parallel or approximately parallel to a plane including the top opening 41).

[0020] The top sealing portion 44 is provided with a top display portion 45, which is simply depicted as "ABC" in the examples of FIGS. 3A to 3C. The side connection portion 43 is provided with a side display portion 46, which is simply depicted as "XYZ" in the examples of FIGS. 3A to 3C. For convenience of drawing, the top display portion 45 and the side display portion 46 are depicted in a normal font in FIGS. 3A to 3C. However, in reality, the top display portion 45 and the side display portion 46 are distorted in images depending on the shape and curvature of the top sealing portion 44 and the side connection portion 43 of the container 40.

[0021] 3A shows, for example, a small lactic acid bacteria beverage container 40, with an upper open portion 41 having a small diameter circle and a lower bottom portion 42 having a large diameter circle, and a side connection portion 43 having a cylindrical shape that expands in diameter from the upper open portion 41 toward the lower bottom portion 42, then becomes a fixed diameter, then becomes depressed once, and then becomes a fixed diameter again. The upper sealing portion 44 is made of a film member (a foil) that is placed over and adhered to the upper open portion 41, and by pinching and peeling off the peripheral portion of the film member (the portion that hangs down outside the upper open portion 41), the upper sealing portion 44 is removed and the upper open portion 41 is exposed.

[0022] 3B shows, for example, a container 40 for cup noodles (cup ramen, cup soba, cup udon, etc.), in which an upper open portion 41 is a large-diameter circle, a lower bottom portion 42 is a small-diameter circle, and a side connection portion 43 is a cylinder whose diameter decreases from the upper open portion 41 toward the lower bottom portion 42. The upper sealing portion 44 is made of a lid member (a flap) that is placed over and adhered to the upper open portion 41, and by pinching and peeling off the peripheral portion of the lid member (the portion that hangs down outside the upper open portion 41), the upper sealing portion 44 is removed and the upper open portion 41 is exposed.

[0023] 3C shows, for example, a PET bottle-type container 40 for beverages (mineral water, tea, soft drinks, etc.), with a top open portion 41 that is a small-diameter circle, a bottom bottom portion 42 that is a large-diameter circle, and a side connection portion 43 that is a cylinder whose diameter increases from the top open portion 41 toward the bottom bottom portion 42 before becoming a constant diameter. The top sealing portion 44 is made up of a cap portion having a threaded portion that screws into a threaded portion formed on the periphery of the top open portion 41, and by gripping and turning the cap portion, the top sealing portion 44 is removed to expose the top open portion 41.

[0024] The shape of the container 40 depicted in FIGS. 3A to 3C is merely an example, and various other shapes of containers 40 are possible. The shapes and configurations of the top open portion 41, bottom bottom portion 42, side connection portion 43, and top sealing portion 44 have a degree of freedom, allowing for various design modifications. For example, the top open portion 41, bottom bottom portion 42, and top sealing portion 44 may be circular and have the same or different diameters, or may have a shape other than a circle (however, a symmetrical shape that allows a center point to be determined is preferable). Furthermore, the side connection portion 43 may have a shape in which the center of the height direction is more bulging than the top or bottom, like a beer keg (a Western-style barrel with a bulging center for filling beer). The contents of the container 40 have a degree of freedom, allowing for various design modifications (it can contain any contents other than food or beverage).

[0025] 4A to 4C are diagrams showing an example of the concept of a container image analysis application service based on an image including a container. The container image analysis application service creates brand fans based on the experience of capturing a container with a camera, while providing benefits (value-added experiences). To receive the container image analysis application service, the user must confirm and agree to the terms of use and privacy policy via the user terminal 10, and then indicate their intention to participate. The container image analysis application service also includes settings such as the campaign name, application period, target products (such as the type and size of the container), prizes, number of winners, and required points.

[0026] In Fig. 4A, the image capturing unit 11 of the user terminal 10 captures (scans) an image of the container before the user eats or drinks from it, and transmits the captured image to the image processing device 20. Here, the image capturing and transmission of the container image in Fig. 4A may be limited in terms of time and number of times (for example, once a day).

[0027] In Fig. 4B, when the image of the container in Fig. 4A is captured and transmitted a predetermined number of times, an automatic application for original goods (prize) is made. In the example of Fig. 4B, capturing and transmitting the image of the container in Fig. 4A is converted into 1 point, and every time 5 points are accumulated, an automatic application for original goods (prize) is made. Here, the automatic application for every 5 points may be limited in terms of time and number of times (for example, up to 10 times during the campaign period).

[0028] In Figure 4C, if a user wins an automatic entry every 5 points, they will win (be provided with) original goods. Figure 4C shows an example of an original good, which is a mobile phone case (original smartphone case) with the container manufacturer XYZ and the product name ABC printed on it.

[0029] The above-described container image analysis application service is intended to promote sales of containers (related products), and therefore, by its nature, it is assumed (presumed) that the containers associated with the container images sent to the image processing device 20 are not the same. In other words, it is assumed (presumed) that each of the predetermined points (e.g., 5 points) required for automatic application for original goods cannot be obtained by multiple images of the same container or images picked up from the Internet, but rather by images taken each time the user eats or drinks (before eating or drinking) the contents of the containers from which the user will consume the contents. Furthermore, it is assumed (presumed) that the containers from which the user will consume the contents have been purchased by, for example, the user, the user's family or housemates, or friends or colleagues who will consume the contents of the containers together with the user.

[0030] The processing device, processing method, and processing program for images including containers disclosed herein, from the perspective of solidifying the above assumptions and presuppositions, are equipped with components that can process (analyze) containers included in an image to determine whether the containers included in the image are the same or are even able to connect them to "proof of purchase" for the container. For example, smooth and reliable proof of purchase would make it easier to create loyalty programs (marketing measures in which companies award rewards to "good customers" who repeatedly use services or frequently purchase products) even for in-store purchases, and could elevate digital sales promotion to a marketing customer relationship process (CRP) for increasing brand fan bases.

[0031] Here, the following factors make it difficult to obtain smooth and reliable proof of purchase: (1) Proof of purchase cannot be mistaken when it comes to handling complaints and fraud. (2) Postcards are not widely distributed and are discarded each time, which is unbearable. (3) Serial proof (serial number) is handled on the factory production line and requires issuing barcodes and printing stickers, resulting in a huge number of items. (4) Receipts are complicated and time-consuming, and there are costs involved in determining whether they are correct, and there are loopholes for fraud. In the past, there have been cases of organized receipt proof of purchase fraud (falsifying the date, folding the receipt to make a different payment, changing the background color, stacking multiple receipts) occurring during sales promotion campaigns. (5) Electronic payment methods such as “○○Pay,” which are less prone to errors, have payment restrictions, are limited to retailers, and lack flexibility due to being driven by platform providers (businesses that provide the infrastructure).

[0032] Thus, there has been a need for a system that allows a container manufacturer (in-house) to process (analyze) the containers in an image to determine whether the containers in the image are the same or not, and even to link them to proof of purchase for the container. Based on this insight, the present inventor has realized a processing device, processing method, and processing program for images containing containers that have such a system. This system shifts the concept of campaign proof of purchase from a fundamentally evil to a fundamentally good one, and includes the flexibility to award slightly excessive points as long as obvious irregularities or fraud are detected. It is based on the possibility of so-called "pseudo proof of purchase," which does not require strict purchase determination.

[0033] 5 is a diagram showing an example of the configuration of image processing device 20. Image processing device 20 has an image input unit (image acquisition unit) 21, a first determination unit 22, a second determination unit 23, a third determination unit 24, an angle calculation unit 25, an angle output unit 26, a comparison unit 27, a notification unit 28, and a bonus granting unit 29.

[0034] The image input unit (image acquisition unit) 21 inputs (acquires) an image including a container that has been captured by the imaging unit 11 of the user terminal 10 and transmitted from the image transmission unit 12. The image input unit (image acquisition unit) 21 inputs (acquires) multiple images from the same user terminal 10.

[0035] The first determination unit 22 analyzes the image input to the image input unit 21 and determines whether or not the container included in the image is a predetermined brand. This determination may be the same as the process used to determine whether or not to capture an image of a container using the imaging unit 11 of the user terminal 10. The first determination unit 22 may calculate the probability (reliability score) that the container included in the image input to the image input unit 21 is a predetermined brand.

[0036] The second determination unit 23 analyzes the image input to the image input unit 21 and determines whether the container included in the image has a predetermined capacity. For example, the second determination unit 23 may determine whether the container included in the image input to the image input unit 21 has a first capacity, a second capacity, or another capacity. The second determination unit 23 may calculate the probability (reliability score) of whether the container included in the image input to the image input unit 21 has the first capacity, the second capacity, or another capacity.

[0037] The third determination unit 24 analyzes the image input to the image input unit 21 and determines whether the image is an existing image lacking originality. The third determination unit 24 detects whether the image input to the image input unit 21 is an image picked up from the Internet (i.e., an image that is highly likely not taken by the user himself / herself via the user terminal 10). The third determination unit 24 uses template matching to compare the image with an Internet image sent to the backend (a part invisible to the user, such as a server-side or database system) and measure the similarity between the two. The third determination unit 24 may calculate the probability (reliability score) that the image input to the image input unit 21 is an existing image lacking originality.

[0038] Angle calculation unit 25 analyzes the image input to image input unit 21 to determine a first line and a second line set based on the contour characteristic parts of the container, and calculates the angle formed by the first line and the second line. In the present disclosure, the first line and the second line are defined by using top surface side display unit 45 provided on top surface sealing unit 44 and side surface side display unit 46 provided on side surface connecting unit 43 as contour characteristic parts of container 40. This will be explained in detail below.

[0039] 6 is a diagram showing an example of the configuration of the angle calculation unit 25. The angle calculation unit has a first line acquisition unit 25X, a second line acquisition unit 25Y, and an angle acquisition unit 25Z.

[0040] The first line acquisition unit 25X acquires a first line based on the upper surface display unit 45 provided on the upper surface sealing unit 44 from the image input to the image input unit 21.

[0041] 7A, 7B, and 7C are diagrams showing an example of the first line acquisition process performed by the first line acquisition unit 25X.

[0042] As shown in FIG. 7A, the first line acquisition unit 25X corrects the top sealing portion 44, which is captured in an image input to the image input unit 21, from a non-circular shape (e.g., an ellipse) to a perfect circle. At this time, distortion of the top display unit 45 provided on the top sealing portion 44 is corrected. Note that if the container 40 is captured from directly above and the top sealing portion 44 in the image is a perfect circle, the correction process of FIG. 7A is not necessary. The correction process of FIG. 7A can also be performed after the first line acquisition and update process of FIGS. 7B and 7C. For example, if the top sealing portion 44 is captured in an elliptical or approximately elliptical shape, the center point of the ellipse or approximately ellipse, which is the intersection of the major axis and minor axis, corresponds to the "center point of the top sealing portion 44."

[0043] As shown in Fig. 7A, the upper surface display unit 45 provided on the upper surface sealing unit 44 is configured to include the character string "ABC" repeatedly arranged in a predetermined column direction and row direction. In the example of Fig. 7B, the character string "ABC" is written in the left-right direction (column direction) in the figure, and the character string "ABC" is repeatedly written in the up-down direction (row direction) in the figure. A predetermined space is provided between the character strings "ABC" in the up-down direction (row direction).

[0044] 7B, the first line acquisition unit 25X acquires, as the first line, a line along the character string "ABC," more specifically, a line drawn by a dashed dotted line along the topmost character string "ABC," in the top display unit 45 provided on the top sealing unit 44. And / or, the first line acquisition unit 25X acquires, as the first line, a line along the boundary between adjacent character strings "ABC," more specifically, a line drawn by a dashed dotted line along the boundary between the bottommost character string "ABC" and the character string "ABC" above it, in the top display unit 45 provided on the top sealing unit 44.

[0045] As shown in Fig. 7C, the first line acquisition unit 25X may acquire an updated version of the first line by translating the line (dashed line) along the character string "ABC" acquired in Fig. 7B or the line (dashed line) along the boundary between adjacent character strings "ABC" so that it passes through the center point of the top sealing portion 44 (the center point of the perfect circle). The updated first line is depicted by a dashed line. The update process in Fig. 7C may be omitted.

[0046] The first line acquisition unit 25X may vary the acquisition and update processes of the first line depending on the shape of the top surface sealing unit 44 captured in the image input to the image input unit 21. For example, if the top surface sealing unit 44 captured in the image is a perfect circle, the updated version of the first line may be acquired by translating a line (dash-dotted line) along the character string "ABC" or a line (dash-two dotted line) along the boundary between adjacent character strings "ABC" so that it passes through the center point of the top surface sealing unit 44 (the center point of the perfect circle). On the other hand, if the top surface sealing unit 44 captured in the image is an oval, the updated version of the first line may be acquired by translating a line (dash-dotted line) along the character string "ABC" or a line (dash-two dotted line) along the boundary between adjacent character strings "ABC" so that it passes through the center point of the major axis or minor axis of the oval of the top surface sealing unit 44.

[0047] There is a degree of freedom in the specific manner in which the first line acquisition unit 25X acquires and updates the first line based on the top surface side display unit 45, and various design modifications are possible. For example, a line along a character string included in the top surface side display unit 45, or a line along a boundary line between adjacent character strings, rotated in the same direction (e.g., clockwise or counterclockwise) by a predetermined angle (e.g., 90° or 45°) may be acquired as the first line. The first line may be acquired and updated according to a common (identical) rule based on the top surface side display unit 45.

[0048] The second line acquisition unit 25Y acquires a second line based on the side display unit 46 provided on the side connection unit 43 from the image input to the image input unit 21.

[0049] The side display unit 46 is configured to include a reference point provided on the side connector 43. This "reference point" is set, for example, as a part of the character string "XYZ" provided on the side connector 43 (side display unit 46). Here, the center (center point) of the letter "Y" is set as the "reference point provided on the side connector 43 (side display unit 46)." As will be described in more detail below, the "reference point provided on the side connector 43 (side display unit 46)" is extracted and determined using segmentation and keypoint extraction (particularly keypoint extraction), with a point on the logo on the side of the container (for example, the center of a part of the character string) as the target. This also applies to the extraction and determination of the character string "ABC" provided on the top display unit 45 and the straight line along the character string.

[0050] FIG. 8 is a diagram illustrating an example of a process for acquiring a second line by the second line acquisition unit 25Y. As shown in FIG. 8, the center point A of the top surface sealing unit 44 is acquired (set, extracted). A reference point provided on the side surface connection unit 43 (side surface display unit 46), i.e., point B where a line extending perpendicularly from the center (center point) of the letter "Y" toward a plane including the top surface sealing unit 44 intersects with the plane including the top surface sealing unit 44, is also acquired (set, extracted). The line connecting points A and B is then acquired as the second line. In this manner, the second line acquisition unit 25Y acquires, from the image input to the image input unit 21, a line connecting the center point A of the top surface sealing unit 44 and point B where a line extending perpendicularly from the reference point of the side surface display unit 46 toward a plane including the top surface sealing unit 44 intersects with the plane including the top surface sealing unit 44.

[0051] In the examples of FIGS. 7A to 7C and 8, the top surface of the container (top sealing portion 44) is circular (approximately circular) in plan view. Therefore, when capturing an image of the container, if the image is captured from directly above the container, the top surface of the container (top sealing portion 44) will appear circular (approximately circular). However, if the image is captured from even a slight angle, the top surface of the container (top sealing portion 44) will appear non-circular, and the latter case is likely to be the most common. Therefore, the first line acquisition unit 25X may correct the top surface of the container (top sealing portion 44) captured in the image from a non-circular shape to a perfect circle before acquiring the first line, and the second line acquisition unit 25Y may correct the top surface of the container (top sealing portion 44) captured in the image from a non-circular shape to a perfect circle before acquiring the second line.

[0052] Angle acquisition unit 25Z acquires the angle formed between the first line acquired by first line acquisition unit 25X and the second line acquired by second line acquisition unit 25Y. The first line acquired in Figures 7A to 7C and the second line acquired in Figure 8 are located on the same plane along the top surface of top surface sealing unit 44, so it is possible to acquire the angle formed between the first line and the second line. Figure 9 is a diagram showing the angle θ formed between the first line and the second line acquired by angle acquisition unit 25Z.

[0053] The angle calculation by the angle calculation unit 25 (first line acquisition unit 25X, second line acquisition unit 25Y, angle acquisition unit 25Z) is performed through three processes: segmentation, keypoint extraction, and angle calculation through geometric processing. That is, when an image including a container is sent to the backend (a part invisible to the user, such as a server side or database system), the three processes of segmentation, keypoint extraction, and angle calculation through geometric processing are performed, resulting in the determination of a first line and a second line, and the angle between the first line and the second line being obtained. In this way, the first line acquisition unit 25X and the second line acquisition unit 25Y acquire the first line and the second line based on the segmentation and keypoint extraction.

[0054] Segmentation is a model that estimates what is in an image for each pixel, and additional learning can be performed by creating original training data using a model architecture used in general object detection. Segmentation determines, on a pixel-by-pixel basis, the area of the target image that is the top surface of the container.

[0055] Keypoint extraction is a model that detects the coordinates of points that characterize an object in an image. New keypoints can be set for containers in the image, allowing for model-based learning. Keypoint extraction detects the coordinates of points in the target image that define (identify) the first and second lines. For example, the coordinates of the center point of the top sealing portion 44 and the plane including the top surface of the top sealing portion 44 are detected to define (identify) the first and second lines. The coordinates of the "ABC" character string that defines (identifies) the first line are also detected. The coordinates of the "XYZ" character string and the reference point within the character string (for example, the center point of the letter "Y") that defines (identifies) the second line are also detected.

[0056] In angle calculation using geometric processing, the angle θ between the first and second lines is calculated based on the data detected by segmentation and keypoint extraction (see Figure 9).

[0057] For example, segmentation takes an image that includes a container as input and determines the location of the top area of the container on a pixel-by-pixel basis. Keypoint extraction takes an image that includes a container as input and identifies points to define (identify) the first and second lines. Angle calculation through geometric processing takes the results of segmentation and keypoint extraction as input and determines the first and second lines, and obtains the angle θ between them.

[0058] When multiple images are input to the image input unit 21 from the same user terminal 10, the angle calculation unit 25 (first line acquisition unit 25X, second line acquisition unit 25Y, angle acquisition unit 25Z) calculates the angle θ between the first line and the second line for each of the multiple images.

[0059] Angle output unit 26 outputs the angle θ formed by the first line and the second line calculated by angle calculation unit 25 (first line acquisition unit 25X, second line acquisition unit 25Y, angle acquisition unit 25Z). When multiple images from the same user terminal 10 are input to image input unit 21 and angle calculation unit 25 calculates the angle θ formed by the first line and the second line for the multiple images, angle output unit 26 outputs the angle θ formed by the first line and the second line for the multiple images.

[0060] The comparison unit 27 compares a plurality of images from the same user terminal 10 to determine whether the angle θ formed by the first line and the second line is within a predetermined range.

[0061] During the manufacture of containers, the positional relationship between the top sealing portion (the top display portion, e.g., the character string "ABC") that seals the top opening of the container and the side connection portion (the side display portion, e.g., the character string "XYZ") of the container is not uniformly manufactured, and the positional relationship between the two can be considered to be the degree of originality of each individual container. In other words, strictly speaking, the angle θ between the first line and the second line will almost always be different for each individual container, and the likelihood that multiple images contain the same container can be determined based on this angle θ. For example, if the angle θ between the first line and the second line is significantly different for containers contained in multiple images, the containers contained in the multiple images are likely to be different. However, if the angle θ between the first line and the second line is the same or nearly the same, the containers contained in the multiple images are likely to be the same.

[0062] For example, the comparison unit 27 calculates the angle θ between the first line and the second line in any one of the reference images as a reference angle θ standard and the angle θ between the first line and the second line in the other reference images of the multiple images is set as the reference angle θ reference Then, the reference angle θ standard and the reference angle θ reference is compared to see if it is within a predetermined range. For example, 0.95 × reference angle θ reference ≦Reference angle θ standard≦1.05×reference angle θ reference If the conditional expression is not satisfied, it can be determined that the containers included in the reference image and the standard image are different. The comparison unit 27 may calculate a probability (reliability score) indicating whether the containers included in the multiple images (standard image and reference image) are the same or not.

[0063] When the angle θ between the first line and the second line for multiple images from the same user terminal 10 is within a predetermined range, that is, when there is a high possibility that the containers included in the multiple images are the same, the notification unit 28 notifies the user terminal 10 of this fact. The notification method for the user terminal 10 can be, for example, a display on a display and / or a voice from a speaker. In this case, the notification unit 28 may notify that the multiple images are not eligible for a bonus by the bonus granting unit 29, which will be described later.

[0064] In addition, if the area of the container in the image including the container input to the image input unit 21 is less than a predetermined range, the notification unit 28 may issue an error notification, stating that it is unable to calculate (obtain) the angle θ formed by the first line and the second line.

[0065] When the angle θ formed by the first line and the second line for multiple images from the same user terminal 10 is not within a predetermined range, the reward granting unit 29 determines that the containers included in the multiple images are different, and grants a reward to the user terminal 10 according to the number of multiple images. For example, each image that includes a container may be converted into 1 point, and when the number of points reaches 5, corresponding to 5 images, an automatic application may be made, and if the user wins the lottery, they may win original goods (see FIGS. 4B and 4C).

[0066] FIG. 10 is a diagram illustrating an example of the concept of a container image analysis application service based on an image containing a container. As shown in FIG. 10, an image containing a container is sent from an application on a user terminal 10 to an AI (artificial intelligence) in an image processing device 20. The AI in the image processing device 20 assigns container information to the uploaded image. More specifically, information such as the angle θ between the first and second lines for each container, the similarity with images on the Internet, the angle θ between the first and second lines for a container most recently uploaded by the same user, and product information (e.g., the container's capacity) can be obtained. The results of the determination by the AI in the image processing device 20 are passed as numerical values to a campaign system, and the campaign system is configured to be able to regulate point and bonus awarding.

[0067] FIG. 11 is a diagram showing a first example of the flow of a method for processing an image including a container.

[0068] In step ST1, the camera serving as the imaging unit 11 of the user terminal 10 is activated. In step ST2, an image including a container is captured (obtained) by the imaging unit 11 of the user terminal 10. When the camera is activated and recognizes a logo or a specific part of the container (for example, characters or a pattern indicating the product name) (when the probability that the container is the target container exceeds a predetermined value), the shutter is automatically released to capture an image including the container.

[0069] In step ST3, the second determination unit 23 of the image processing device 20 determines whether the container has a predetermined capacity. This determination may be made, for example, by distinguishing the capacity from the overall shape of the container, or by using the product ID (JAN code) attached to the container. In step ST4, the third determination unit 24 of the image processing device 20 confirms that the image including the container is not an existing image (an image picked up from the Internet) that lacks originality.

[0070] In step ST5, the angle calculation unit 25 (first line acquisition unit 25X, second line acquisition unit 25Y, angle acquisition unit 25Z) of the image processing device 20 calculates the angle formed by the first line and the second line for a container included in the captured image. In step ST6, the comparison unit 27 of the image processing device 20 compares the angle formed by the first line and the second line of a container included in an image most recently registered by a user with the same ID (the same user) with the angle formed by the first line and the second line calculated in step ST5, by checking against a database, and determines the degree of match. In step ST7, a final determination is made as to whether the containers included in the multiple images whose angles have been compared and determined in step ST6 are the same container or different containers.

[0071] Figures 12A, 12B, 12C, and 12D are diagrams showing a second example of the flow of a method for processing an image including a container. Figures 12A to 12D depict a container of the type illustrated in Figure 3A.

[0072] In FIG. 12A, filter 1, which uses a camera AI model, determines whether the container is a specified brand. In FIG. 12B, filter 2, which uses a back-end AI model (object detection), determines whether the container is captured at a certain size and the character string "XYZ" on the side display. In FIG. 12C, filter 3, which uses a back-end AI model (keypoint detection), determines the character string "ABC" on the top display and also determines the reference point (e.g., the center of the central "Y") of the character string "XYZ" on the side display. In FIG. 12D, filter 4, which uses a back-end AI model (segmentation), performs ellipse detection for angle calculation from the top sealing part of the container and obtains the angle between the first line obtained based on the character string "ABC" on the top display and the second line obtained based on the character string "XYZ" on the side display.

[0073] In this manner, in the image processing device for images including containers according to the present disclosure, the first line acquisition unit acquires a first line based on the top display unit from the image including the container, the second line acquisition unit acquires a second line based on the side display unit from the image including the container, and the angle acquisition unit acquires the angle between the first line and the second line. This enables the processing of images including suitable (improved) containers. For example, analysis of the containers included in the image can be performed to determine whether the containers included in the image are the same, and can even be used to provide proof of purchase for the containers. Furthermore, the system is flexible, convenient, and can achieve a certain level of accuracy.

[0074] Furthermore, the image processing device for images including containers according to the present disclosure maintains a probability of whether an image contains a specific container or whether it is the same container, ensuring flexibility in determining whether the image is correct or incorrect. Furthermore, detailed feedback of metadata related to the determination allows for flexible changes in campaign design. Furthermore, since analysis processing is performed on the application server side (image processing device 20) and container recognition is performed on the application side (user terminal 10), a user experience (UX) such as a real-time feel can be achieved. Furthermore, implementation is easy, as it can be achieved using, for example, JavaScript (registered trademark) running on the HTML language.

[0075] Fig. 13 is a diagram showing an example of the functional configuration of a user terminal 10 and an image processing device 20 according to an embodiment. As shown in Fig. 13, the user terminal 10 and the image processing device 20 have a control unit 110, a storage unit 120, a communication unit 130, an input unit 140, and an output unit 150. Note that Fig. 13 shows an example of functional blocks of the user terminal 10 and the image processing device 20, and the user terminal 10 and the image processing device 20 may have other functional blocks not shown in Fig. 13. Furthermore, the configuration may not include some of the functional blocks.

[0076] The control unit 110 controls the user terminal 10 and the image processing device 20. The control unit 110 can be configured by a controller, a control circuit, or a control device that is described based on common understanding in the technical field related to the present invention.

[0077] The storage unit 120 stores information used by the user terminal 10 and the image processing device 20. The storage unit 120 can be configured, for example, by a memory, storage, storage device, or the like that is described based on common understanding in the technical field related to the present invention.

[0078] The communication unit 130 performs communication between the user terminal 10 and the image processing device 20 (e.g., mutual communication between the two). The communication unit 130 can be configured by a transmitter / receiver, a transmission / reception circuit, or a transmission / reception device, which are described based on common understanding in the technical field related to the present invention. Note that the communication unit 130 may be configured by a transmission unit and a reception unit.

[0079] The input unit 140 accepts inputs to the user terminal 10 and the image processing device 20. The input unit 140 may also be connected to a predetermined device, storage medium, etc., and accept data input. The input unit 140 may output the input results to, for example, the control unit 110. The input unit 140 may be configured with input devices such as a keyboard, mouse, and buttons, input / output terminals, input / output circuits, etc., which are explained based on common understanding in the technical field related to the present invention. The input unit 140 may also be configured as an integrated unit with a display unit (for example, a touch panel).

[0080] The output unit 150 performs output in the user terminal 10 and the image processing device 20. For example, the output unit 150 may be configured to include a display unit that displays images, an audio output unit that outputs audio, etc. The display unit may be configured, for example, by a display device such as a monitor or the like that is described based on common understanding in the technical field related to the present invention. Furthermore, the audio output unit may be configured by an output device such as a speaker that is described based on common understanding in the technical field related to the present invention.

[0081] The block diagrams used to explain the above embodiments show functional blocks. These functional blocks (components) are realized by any combination of hardware and / or software. Furthermore, the means for realizing each functional block is not particularly limited. That is, each functional block may be realized by a single physically coupled device, or may be realized by two or more physically separate devices connected by wire or wirelessly.

[0082] For example, the user terminal 10 and the image processing device 20 according to an embodiment of the present disclosure may function as a computer that processes the processing method and processing program for an image including a container according to the present disclosure. Fig. 15 is a diagram illustrating an example of the hardware configuration of the user terminal 10 and the image processing device 20 according to an embodiment. The above-described user terminal 10 and the image processing device 20 may be physically configured as a computer device including a processor 1001, a memory 1002, a storage 1003, a communication device 1004, an input device 1005, an output device 1006, a bus 1007, etc.

[0083] In this disclosure, terms such as apparatus, circuit, device, unit, and server may be used interchangeably. The hardware configuration of the user terminal 10 and the image processing device 20 may be configured to include one or more of the devices shown in the drawings, or may be configured to exclude some of the devices.

[0084] For example, although only one processor 1001 is shown, there may be multiple processors. Furthermore, processing may be performed by one processor, or processing may be performed by two or more processors simultaneously, serially, or in other ways. Furthermore, processor 1001 may be implemented by one or more chips.

[0085] Each function of the user terminal 10 and the image processing device 20 is realized by loading specified software (programs) onto hardware such as the processor 1001, memory 1002, etc., so that the processor 1001 performs calculations and controls communication via the communication device 1004, reading and / or writing of data in the memory 1002 and storage 1003, etc.

[0086] The processor 1001 controls the entire computer by running, for example, an operating system. The processor 1001 may be configured as a central processing unit (CPU) including an interface with peripheral devices, a control device, an arithmetic unit, a register, etc. Note that each unit such as the control unit 110 described above may be realized by the processor 1001.

[0087] The processor 1001 also reads programs (program codes), software modules, data, etc. from at least one of the storage 1003 and the communication device 1004 into the memory 1002, and executes various processes in accordance with these. The programs used are those that cause a computer to execute at least some of the operations described in the above-described embodiments. For example, the control unit 110 may be realized by a control program stored in the memory 1002 and running on the processor 1001, and similar implementations may be made for other functional blocks.

[0088] The memory 1002 is a computer-readable recording medium and may be configured by at least one of, for example, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), an electrically EEPROM (EEPROM), a random access memory (RAM), or other suitable storage medium. The memory 1002 may also be referred to as a register, a cache, a main memory, or the like. The memory 1002 may store executable programs (program codes), software modules, and the like for implementing a method according to one embodiment.

[0089] Storage 1003 is a computer-readable recording medium, and may be configured by at least one of, for example, a flexible disk, a floppy disk, a magneto-optical disk (e.g., a compact disk (CD-ROM (Compact Disc ROM)), a digital versatile disk, a Blu-ray (registered trademark) disk), a removable disk, a hard disk drive, a smart card, a flash memory device (e.g., a card, a stick, a key drive), a magnetic stripe, a database, a server, or other suitable storage medium. Storage 1003 may also be called an auxiliary storage device. Note that the above-mentioned storage unit 120 may be realized by memory 1002 and / or storage 1003.

[0090] The communication device 1004 is hardware (transmission / reception device) for communicating between computers via at least one of a wired network and a wireless network, and is also referred to as a network device, a network controller, a network card, a communication module, etc. The communication device 1004 may include a SIM card. Note that the above-mentioned communication unit 130 may be realized by the communication device 1004.

[0091] The input device 1005 is an input device (for example, a keyboard, a mouse, etc.) that receives input from the outside. The output device 1006 is an output device (for example, a display, a speaker, etc.) that performs output to the outside. The input device 1005 and the output device 1006 may be integrated into one unit (for example, a touch panel). The above-mentioned input unit 140 and output unit 150 may be realized by the input device 1005 and the output device 1006, respectively.

[0092] Furthermore, each device such as the processor 1001 and the memory 1002 is connected by a bus 1008 for communicating information. The bus 1008 may be configured as a single bus, or may be configured as different buses between the devices.

[0093] Furthermore, the user terminal 10 and the image processing device 20 may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field programmable gate array (FPGA), and some or all of the functional blocks may be realized by the hardware. For example, the processor 1001 may be implemented by at least one of these pieces of hardware.

[0094] (Variation) In addition, terms explained in this disclosure and / or terms necessary for understanding this disclosure may be replaced with terms having the same or similar meanings.

[0095] The information, parameters, etc. described in this disclosure may be expressed using absolute values, relative values from a predetermined value, or other corresponding information. Furthermore, the names used for parameters, etc. in this disclosure are not limiting in any way.

[0096] The information, signals, etc. described in this disclosure may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

[0097] Information, signals, etc. may be input / output via multiple network nodes. The input / output information, signals, etc. may be stored in a specific location (e.g., memory) or may be managed using a table. The input / output information, signals, etc. may be overwritten, updated, or added. Output information, signals, etc. may be deleted. Input information, signals, etc. may be transmitted to another device.

[0098] Furthermore, notification of specified information (e.g., notification that "it is X") is not limited to explicit notification, but may be made implicitly (e.g., by not notifying the specified information or by notifying other information).

[0099] Software shall be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise.

[0100] Software, instructions, information, etc. may also be transmitted and received via a transmission medium and / or signal waveform. For example, if software is transmitted from a website, server, or other remote source using wired technologies (such as coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL)), and / or wireless technologies (such as infrared, microwave), then these wired and / or wireless technologies are included within the definition of transmission media.

[0101] As used in this disclosure, the terms "system" and "network" may be used interchangeably.

[0102] Each aspect / embodiment described in this disclosure may be used alone, in combination, or switched depending on the implementation. Furthermore, the order of the processing procedures, sequences, flowcharts, etc. of each aspect / embodiment described in this disclosure may be changed unless inconsistent. For example, the methods described in this disclosure present elements of various steps using an example order, and are not limited to the specific order presented.

[0103] As used in this disclosure, the phrase "based on" does not mean "based only on," unless expressly stated otherwise. In other words, the phrase "based on" means both "based only on" and "based at least on."

[0104] As used in this disclosure, any reference to an element using a designation such as "first," "second," etc. does not generally limit the quantity or order of those elements. These designations may be used in this disclosure as a convenient method of distinguishing between two or more elements. Thus, a reference to a first and a second element does not imply that only two elements may be employed or that the first element must in some way precede the second element.

[0105] When used in this disclosure, the terms "include," "including," and variations thereof are intended to be inclusive, similar to the term "comprising." Furthermore, when used in this disclosure, the term "or" is not intended to be an exclusive or.

[0106] In this disclosure, where articles are added by translation, such as a, an, and the in English, the disclosure may include that the nouns following these articles are in the plural form.

[0107] Although the invention according to the present disclosure has been described in detail above, it is clear to those skilled in the art that the invention according to the present disclosure is not limited to the embodiments described in the present disclosure. The invention according to the present disclosure can be implemented in modified and altered forms without departing from the spirit and scope of the invention as defined by the description of the claims. Therefore, the description of the present disclosure is intended to be illustrative and does not impose any limiting meaning on the invention according to the present disclosure. [Explanation of symbols]

[0108] 1 System 10 User terminal 11 Imaging unit (image acquisition unit) 12 Image transmission unit 13 Container Image Analysis Application Department 20 Image processing device including container (image processing device) 21 Image input unit (image acquisition unit) 22 First Judgment Section 23 Second Judgment Section 24 Third Judgment Section 25 Angle calculation unit 25X 1st straight line acquisition part 25Y 2nd straight line acquisition part 25Z angle acquisition part 26 Angle output unit 27 Comparison section 28 Notification Department 29 Benefits Granting Department 30 Network 40 containers 41 Open top part 42 Bottom bottom 43 Side connection 44 Top sealing part 45 Top side display section 46 Side display section

Claims

1. An open top portion; A bottom surface and a side connection portion connecting the upper surface opening portion and the lower surface bottom portion; an upper surface sealing portion that seals the upper surface opening portion; an upper display portion including a character string provided on the upper sealing portion; a side display unit including a reference point provided on the side connection unit; An image processing device including a container having an image acquisition unit that acquires an image including the container, the image being captured by an imaging unit of the user terminal and transmitted by an image transmission unit of the user terminal; a first line acquisition unit that acquires a first line based on the upper surface display unit from the image; a second line acquisition unit that acquires a second line based on the side display unit from the image; an angle acquisition unit that acquires an angle between the first straight line and the second straight line; and the first straight line acquisition unit acquires, as the first straight line, a straight line along the character string or a straight line along a boundary line between the adjacent character strings, and acquires, as an updated version of the first straight line, a straight line along the character string or a straight line along the boundary line between the adjacent character strings translated so as to pass through a center point of the top surface sealing portion; the second straight line acquisition unit acquires, as the second straight line, a straight line connecting a center point of the top surface sealing unit and a point at which a straight line extending perpendicularly from the reference point of the side display unit toward a plane including the top surface sealing unit intersects with the plane including the top surface sealing unit. An image processing device including a container.

2. An open top portion; A bottom surface and a side connection portion connecting the upper surface opening portion and the lower surface bottom portion; an upper surface sealing portion that seals the upper surface opening portion; an upper display portion including a character string provided on the upper sealing portion; a side display unit including a reference point provided on the side connection unit; An image processing device including a container having an image acquisition unit that acquires an image including the container, the image being captured by an imaging unit of the user terminal and transmitted by an image transmission unit of the user terminal; a first line acquisition unit that acquires a first line based on the upper surface display unit from the image; a second line acquisition unit that acquires a second line based on the side display unit from the image; an angle acquisition unit that acquires an angle between the first straight line and the second straight line; and the first straight line acquisition unit acquires, as the first straight line, a straight line along the character string or a straight line along the boundary line of the adjacent character strings, and, when the top sealing part reflected in the image is a perfect circle, acquires, as an updated version of the first straight line, a straight line along the character string or a straight line along the boundary line of the adjacent character strings, which is translated so as to pass through a center point of the perfect circle; and, when the top sealing part reflected in the image is an ellipse, acquires, as an updated version of the first straight line, a straight line along the character string or a straight line along the boundary line of the adjacent character strings, which is translated so as to pass through a center point of a major axis or a minor axis of the ellipse; the second straight line acquisition unit acquires, as the second straight line, a straight line connecting a center point of the top surface sealing unit and a point at which a straight line extending perpendicularly from the reference point of the side display unit toward a plane including the top surface sealing unit intersects with the plane including the top surface sealing unit. An image processing device including a container.

3. the first straight line acquisition unit and the second straight line acquisition unit correct the top surface sealing portion captured in the image from a non-circular shape to a perfect circle, and then acquire the first straight line and the second straight line. An image processing device including the container according to claim 1.

4. the first line acquisition unit and the second line acquisition unit acquire the first line and the second line based on segmentation and keypoint extraction; An image processing device including the container according to claim 1 or 2.

5. An open top portion; A bottom surface and a side connection portion connecting the upper surface opening portion and the lower surface bottom portion; an upper surface sealing portion that seals the upper surface opening portion; an upper display portion including a character string provided on the upper sealing portion; a side display unit including a reference point provided on the side connection unit; 1. A method for processing an image including a container having: acquiring an image including the container, the image being captured by an imaging unit of the user terminal and transmitted by an image transmission unit of the user terminal; obtaining a first straight line based on the top display unit from the image; obtaining a second straight line based on the side display unit from the image; obtaining an angle between the first line and the second line; and In the step of acquiring the first straight line, a straight line along the character string or a straight line along a boundary line between the adjacent character strings is acquired as the first straight line, and a straight line along the character string or a straight line along the boundary line between the adjacent character strings is translated so as to pass through a center point of the top sealing portion, and acquired as an updated version of the first straight line; In the step of obtaining the second straight line, a straight line connecting a center point of the top surface sealing portion and a point at which a straight line extending perpendicularly from the reference point of the side display portion toward a plane including the top surface sealing portion intersects with the plane including the top surface sealing portion is obtained as the second straight line. A method for processing an image including a container, characterized in that each of the above processing steps is executed by a computer.

6. An open top portion; A bottom surface and a side connection portion connecting the upper surface opening portion and the lower surface bottom portion; an upper surface sealing portion that seals the upper surface opening portion; an upper display portion including a character string provided on the upper sealing portion; a side display unit including a reference point provided on the side connection unit; 1. A method for processing an image including a container having: acquiring an image including the container, the image being captured by an imaging unit of the user terminal and transmitted by an image transmission unit of the user terminal; obtaining a first straight line based on the top display unit from the image; obtaining a second straight line based on the side display unit from the image; obtaining an angle between the first line and the second line; and In the step of acquiring the first straight line, a straight line along the character string or a straight line along a boundary line between adjacent character strings is acquired as the first straight line, and when the top surface sealing part reflected in the image is a perfect circle, a straight line along the character string or a straight line along the boundary line between the adjacent character strings is translated in parallel so as to pass through a center point of the perfect circle, and an updated version of the first straight line is acquired; when the top surface sealing part reflected in the image is an ellipse, a straight line along the character string or a straight line along the boundary line between the adjacent character strings is translated in parallel so as to pass through a center point of a major axis or a minor axis of the ellipse, and an updated version of the first straight line is acquired; In the step of obtaining the second straight line, a straight line connecting a center point of the top surface sealing portion and a point at which a straight line extending perpendicularly from the reference point of the side display portion toward a plane including the top surface sealing portion intersects with the plane including the top surface sealing portion is obtained as the second straight line. A method for processing an image including a container, characterized in that each of the above processing steps is executed by a computer.

7. An open top portion; A bottom surface and a side connection portion connecting the upper surface opening portion and the lower surface bottom portion; an upper surface sealing portion that seals the upper surface opening portion; an upper display portion including a character string provided on the upper sealing portion; a side display unit including a reference point provided on the side connection unit; A processing program for an image including a container having acquiring an image including the container, the image being captured by an imaging unit of the user terminal and transmitted by an image transmission unit of the user terminal; obtaining a first straight line based on the top display unit from the image; obtaining a second straight line based on the side display unit from the image; obtaining an angle between the first line and the second line; and In the step of acquiring the first straight line, a straight line along the character string or a straight line along a boundary line between the adjacent character strings is acquired as the first straight line, and a straight line along the character string or a straight line along the boundary line between the adjacent character strings is translated so as to pass through a center point of the top sealing portion, and acquired as an updated version of the first straight line; In the step of obtaining the second straight line, a straight line connecting a center point of the top surface sealing portion and a point at which a straight line extending perpendicularly from the reference point of the side display portion toward a plane including the top surface sealing portion intersects with the plane including the top surface sealing portion is obtained as the second straight line. A processing program for an image including a container, which causes a computer to execute each of the above processing steps.

8. An open top portion; A bottom surface and a side connection portion connecting the upper surface opening portion and the lower surface bottom portion; an upper surface sealing portion that seals the upper surface opening portion; an upper display portion including a character string provided on the upper sealing portion; a side display unit including a reference point provided on the side connection unit; A processing program for an image including a container having acquiring an image including the container, the image being captured by an imaging unit of the user terminal and transmitted by an image transmission unit of the user terminal; obtaining a first straight line based on the top display unit from the image; obtaining a second straight line based on the side display unit from the image; obtaining an angle between the first line and the second line; and In the step of acquiring the first straight line, a straight line along the character string or a straight line along a boundary line between adjacent character strings is acquired as the first straight line, and when the top surface sealing part reflected in the image is a perfect circle, a straight line along the character string or a straight line along the boundary line between the adjacent character strings is translated in parallel so as to pass through a center point of the perfect circle, and an updated version of the first straight line is acquired; when the top surface sealing part reflected in the image is an ellipse, a straight line along the character string or a straight line along the boundary line between the adjacent character strings is translated in parallel so as to pass through a center point of a major axis or a minor axis of the ellipse, and an updated version of the first straight line is acquired; In the step of obtaining the second straight line, a straight line connecting a center point of the top surface sealing portion and a point at which a straight line extending perpendicularly from the reference point of the side display portion toward a plane including the top surface sealing portion intersects with the plane including the top surface sealing portion is obtained as the second straight line. A processing program for an image including a container, which causes a computer to execute each of the above processing steps.

Images