Marker processing device and marker sheet

Abstract

To improve or secure the accuracy of information on the position of an imaging unit.SOLUTION: A marker processor includes: an imaging unit for imaging a target surface including a marker and acquiring the taken image; an acquisition unit for acquiring positional information showing the positional relation between the imaging unit and the target surface on the basis of detection of the marker from the image; a storage unit for storing positional information for reference; and a comparison unit for comparing the positional information acquired by the acquisition unit and the positional information for reference stored in the storage unit.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The present invention relates to a marker processing device and a marker sheet. [Background technology]

[0002] Conventionally, delivery service providers' offices, convenience stores, and the like accept parcels for delivery from senders in the following procedure: The sender writes information about the sender, such as the sender's name and address, and information about the recipient, such as the recipient's name and address, on an address label. Then, when a convenience store or other store clerk receives the address label with this information written on it and the parcel to be sent from the sender, the clerk measures the external dimensions and weight of the parcel using a tape measure or scale, and determines and settles the delivery fee based on the measurement results and the recipient's address, completing the delivery acceptance.

[0003] However, in the above-mentioned home delivery reception, since it takes time to measure the external dimensions and weight of the package, it is preferable for the sender to measure the dimensions and weight of the package himself in order to reduce the burden on the store clerk. Furthermore, if the burden on the store clerk can be reduced, it is possible to prevent customers from waiting for a long time at the store's cash register counter, which leads to improved service for customers. Therefore, in recent years, measuring devices have been developed that automatically measure the dimensions and weight of packages without bothering store clerks. Patent Document 1 discloses an example of such a measuring device. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2018-194369 A Summary of the Invention [Problem to be solved by the invention]

[0005] In such a measuring device, it is important that information regarding the relative positional relationship between the imaging unit that images the baggage and the measurement range is correctly stored.

[0006] An object of the present invention is to provide a new and improved marker processing device and marker sheet that can improve or ensure the accuracy of information regarding the position of an imaging unit. [Means for solving the problem]

[0007] In order to solve the above problem, according to one aspect of the present invention, there is provided a method for detecting a positional relationship between the object surface and the imaging unit by detecting the positional relationship between the object surface and the imaging unit; a storage unit that stores reference positional information; and a comparison unit that compares the positional information obtained by the storage unit with the reference positional information stored in the storage unit, the captured image acquired by the imaging unit includes a visible light image and a distance image, and the position information includes first position information indicating a position in the visible light image according to a detection result of the marker, and depth information of a planar position corresponding to the position indicated by the first position information in the distance image as second position information; A marker processing device is provided.

[0010] The reference position information is position information obtained based on detection of the marker from a first captured image obtained by imaging at a first time point, and the comparison unit may compare the reference position information with the position information obtained based on detection of the marker from a second captured image obtained by imaging at a second time point later than the first time point.

[0011] The marker processing device further includes a size measurement unit that measures the size of the object based on the distance image acquired by the imaging unit when the object is placed on the target surface, and the size measurement unit may perform measurement of the size of the object based on a predetermined comparison result being obtained by the comparison unit.

[0012] The predetermined comparison result may be a comparison result indicating that a difference between the location information acquired by the acquisition unit and the reference location information is equal to or smaller than a threshold value.

[0013] If the comparison unit obtains a comparison result indicating that the difference exceeds the threshold, the size measurement unit may not perform measurement of the size of the object.

[0014] The marker processing device may further include a display control unit that generates a measurement result screen that shows a result of the measurement by the size measurement unit.

[0015] The display control unit may generate an adjustment guidance screen that provides guidance for adjusting a position or attitude of the imaging unit when the comparison unit obtains a comparison result indicating that the difference exceeds the threshold value.

[0017] In order to solve the above problem, according to another aspect of the present invention, there is provided a marker formed of a first primary color, the marker having a pattern made of a color not including the first primary color. and the pattern is a collection of irregularly arranged pattern elements. A marker sheet is provided.

[0020] In order to solve the above problems, according to another aspect of the present invention, A magnetic marker sheet is provided that includes a marker formed with a first primary color, the marker having a pattern made of a color that does not include the first primary color.

[0021] In order to solve the above problems, according to another aspect of the present invention, A marker sheet is provided, the marker including a marker formed of a first primary color, the marker having a pattern made of a color not including the first primary color, the marker having an outer shape that is a rectangle having a first side and a second side perpendicular to the first side, and the marker sheet including a first straight line parallel to the first side and a second straight line parallel to the second side that are formed so as to intersect with each other.

[0022] In order to solve the above problem, according to another aspect of the present invention, there is provided an imaging unit that captures an image of a marker sheet that includes markers formed in a first primary color and having a pattern made of a color that does not include the first primary color and that is placed on a target surface to acquire a visible light image and a distance image, a detection unit that detects the position of the marker in the visible light image, and an acquisition unit that acquires depth information of a planar position in the distance image that corresponds to the detection result by the detection unit; a measurement range determination unit that determines a measurement range based on the planar position and the depth information; and a size measurement unit that measures a size of an object by extracting data within the measurement range from the distance image acquired by the imaging unit with the object placed on the target surface. wherein the detection unit detects the position of the marker from a processed image obtained by extracting a second primary color component from the visible light image. Effect of the Invention

[0027] According to the present invention described above, it is possible to improve or ensure the accuracy of information relating to the position of the imaging unit. [Brief description of the drawings]

[0028] [Figure 1] FIG. 1 is an external view of an automatic reception machine 10 according to a first embodiment of the present invention. [Diagram 2] 3 is a flowchart showing an outline of the operation of the automatic reception machine 10 according to the first embodiment of the present invention. [Diagram 3] 2 is an explanatory diagram showing the function of a control unit 150. FIG. [Figure 4] FIG. 2 is an explanatory diagram showing the configuration of a calibration chart 20. [Diagram 5] FIG. 11 is an explanatory diagram showing a specific example of a processed image made up of a red component of an RGB image. [Figure 6] 5 is a flowchart showing a calibration operation according to the first embodiment of the present invention. [Figure 7] FIG. 11 is an explanatory diagram showing the external configuration of an automatic reception machine 12 according to a second embodiment. [Figure 8] FIG. 11 is an explanatory diagram showing functions of a control unit 160 according to the second embodiment of the present invention. [Figure 9] 10 is a flowchart showing an operation during calibration according to the second embodiment. [Figure 10] 10 is a flowchart showing the operation of the automatic reception machine 12 according to the second embodiment when in use. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configurations are denoted by the same reference numerals, and duplicated explanations will be omitted.

[0030] In addition, in this specification and drawings, multiple components having substantially the same functional configuration may be distinguished by adding different alphabets after the same reference numeral. However, when there is no need to particularly distinguish between multiple components having substantially the same functional configuration, each of the multiple components is given only the same reference numeral.

[0031] The embodiment of the present invention described below may be applied to delivery reception at a convenience store, etc. However, the embodiment of the present invention is not limited to use in a convenience store, etc., and may be used in public facilities such as delivery company stores, retail stores, supermarkets, and transportation stations, and is not particularly limited.

[0032] In the following description, the requester refers to a person who brings a package to a convenience store or the like for home delivery, and in the following embodiment, the person who actually requests home delivery and the person who brings the package are described as being the same person. However, in this embodiment, the person who actually requests home delivery and the person who brings the package do not necessarily have to be the same person.

[0033] The automatic reception machine according to the embodiment of the present invention measures the size of an object to be measured, i.e., luggage, brought in by a sender and placed on a placement surface. Any of the functions included in the automatic reception machine described below may be omitted depending on the installation location, the expected sender, etc. On the other hand, the automatic reception machine may also have functions not described below.

[0034] <<1. First embodiment>> <1-1. Overview of the automated reception machine> (External appearance of the automatic reception machine) FIG. 1 is an external view of an automatic reception machine 10 according to a first embodiment of the present invention. The automatic reception machine 10 according to the first embodiment of the present invention has an operation display unit 110, a receipt printer 120, a support unit 130, an imaging unit 140, and a control unit 150. A placement surface F is an example of a target surface imaged by the imaging unit 140, and luggage is placed on the placement surface F. The calibration chart 20 is a marker sheet for calibration on which markers are formed. The calibration chart 20 is placed on the placement surface F when the automatic reception machine 10 or the imaging unit 140 is installed, and is used for calibration to determine a measurement range H expressed by a relative position from the imaging unit 140. In the example shown in FIG. 1, two markers (calibration markers) are formed on the calibration chart 20. From the viewpoint of clarity of the drawing, the markers are shown with diagonal lines in FIG. 1, but in reality, the markers may be two-dimensional codes (see FIG. 4). The calibration chart 20 is removed from the placement surface F after calibration is performed.

[0035] The operation display unit 110 is a touch panel having functions as a display unit and an operation unit. The operation display unit 110 displays a screen mainly for the requester who is the operator, and accepts operations by the requester. The operation display unit 110 displays appropriate text, illustrations and videos of operation guides at each stage of the transaction, and RGB images acquired by the imaging unit 140.

[0036] The receipt printer 120 prints a receipt for the sender before the transaction is completed. The receipt may also include measurements of the size of the package and the delivery charge. If the automated reception machine 10 is equipped with a settlement function such as a cash processing unit, the charge may be settled at the automated reception machine 10. If the automated reception machine 10 is not equipped with a settlement function, the sender may, for example, take the receipt to a cash register and settle the charge there.

[0037] The support section 130 is provided on the placement surface F. The support section 130 includes a vertical region 132 that exists along the up-down direction, and a horizontal region 134 that exists along the horizontal direction from the upper end of the vertical region 132. The support section 130 supports the imaging section 140 at the tip of the horizontal region 134. The imaging section 140 is located, for example, on a vertical line that passes through the center of the measurement range H in a planar view.

[0038] The imaging unit 140 captures an image of the baggage placed on the placement surface F to obtain an image including the baggage. The imaging unit 140 may have a visible light camera that obtains an RGB image as a visible light image, and a depth camera that obtains a distance image. In this specification, the description will be given mainly assuming an example in which the depth camera is a stereo camera that obtains a parallax image, but the depth camera may be a laser-type camera such as a LiDAR (Light Detection and Ranging) camera.

[0039] The control unit 150 controls the overall operation of the automatic reception machine 10. For example, the control unit 150 controls the measurement of the size of the package and the display screen displayed on the operation display unit 110. Furthermore, when the automatic reception machine 10 or the imaging unit 140 is installed, the control unit 150 performs calibration to determine the measurement range H by detecting markers from the calibration chart 20 placed on the placement surface F. In other words, the automatic reception machine 10 having the control unit 150 has a function as a marker processing device. The function of the control unit 150 and the configuration of the calibration chart 20 will be described in detail later.

[0040] (Outline of operation of the automatic reception machine 10) The external configuration of the automatic reception machine 10 according to the first embodiment of the present invention has been described above. Next, the general operation of the automatic reception machine 10 according to the first embodiment of the present invention will be described with reference to FIG.

[0041] 2 is a flowchart showing the general operation of the automatic reception machine 10 according to the first embodiment of the present invention. First, the operation display unit 110 displays a service selection screen (S71). When home delivery of luggage is selected on the service selection screen, the operation display unit 110 displays a display screen that guides the user to place the luggage on the placement surface F (S72).

[0042] After that, the imaging unit 140 acquires a distance image of the placement surface F on which the luggage is placed (S73). The control unit 150 then generates point cloud data from the distance image (S74), and converts the coordinate system of the point cloud data from the camera coordinate system to the coordinate system of the measurement range H (S75). Furthermore, the control unit 150 extracts point cloud data within the measurement range H from the point cloud data, and measures the size of the luggage from the extracted point cloud data (S77). The size of the luggage may be the sum of the dimensions of the length, width, and height of a rectangular parallelepiped circumscribing the luggage, or may be a category to which the above sum belongs among multiple categories prepared in advance.

[0043] Then, the operation display unit 110 displays the measurement results of the package size, and the receipt printer 120 creates a receipt by printing the measurement results, etc. on a paper medium, and ejects the receipt (S78).

[0044] In this way, the automatic reception machine 10 according to the embodiment of the present invention can measure the size of a package without operation by a store clerk or attendant. Therefore, it is possible to reduce congestion at manned counters in stores and reduce the labor required for counter operations. However, the automatic reception machine 10 according to the embodiment of the present invention should not be precluded from being used by a store clerk or attendant, and a store clerk or attendant can also operate the automatic reception machine 10 to measure the size of a package.

[0045] (background) The above-mentioned calibration is a process for determining a measurement range expressed by a relative position from an imaging unit, and is executed, for example, by detecting the position of a marker from an RGB image of a calibration chart and acquiring depth information of a planar position in a distance image corresponding to the detection result. Here, in the calibration chart according to the comparative example, for example, a marker expressed in black is formed on a white background. However, sufficient difference information cannot be obtained from two parallax images obtained by imaging such a calibration chart, and it is difficult to obtain accurate distance images and depth information.

[0046] In this regard, by applying a pattern to the markers or the entire calibration chart, the amount of difference information obtained from the two parallax images increases, and the accuracy of the depth information can also be improved. However, it may be difficult to detect the position of the marker from the RGB image of the calibration chart with the pattern. This may cause an error in the planar position where the depth information should be obtained in the distance image.

[0047] The inventor of the present invention has come to create the first embodiment of the present invention by focusing on the above circumstances. According to the first embodiment of the present invention, it is possible to improve the accuracy of information related to the position of the imaging unit 140, that is, to improve the accuracy of the calibration for determining the measurement range H expressed by the relative position from the imaging unit 140. The functions of the control unit 150 of the automatic reception machine 10 according to the first embodiment will be described in more detail below.

[0048] <1-2. Functions of the control unit> 3 is an explanatory diagram showing the functions of the control unit 150. As shown in FIG. 3, the control unit 150 has a marker detection unit 151, a depth information acquisition unit 152, a measurement range determination unit 153, a storage unit 154, a size measurement unit 155, and a display control unit 156. The marker detection unit 151, the depth information acquisition unit 152, and the measurement range determination unit 153 are functions mainly used when performing calibration for determining the measurement range H, and the size measurement unit 155 is a configuration used when actually measuring the size of a package. The functions of the control unit 150 can be realized by a processor reading a program or software stored in a memory and operating according to the program or software.

[0049] (marker detection unit 151) The marker detection unit 151 detects the positions of markers formed on the calibration chart 20 from the RGB image acquired by the imaging unit 140 when calibration is performed. Below, the configuration of the calibration chart 20 will be described, and then the detection process by the marker detection unit 151 will be described in more detail.

[0050] FIG. 4 is an explanatory diagram showing the configuration of the calibration chart 20. In the example shown in FIG. 4, two calibration markers 21 are formed on the calibration chart 20. The calibration markers 21 have a rectangular outer shape. Furthermore, a first straight line 23 and a second straight line 24 are formed on the calibration chart 20. The first straight line 23 is parallel to one side of the calibration marker 21, and the second straight line 24 is parallel to the other side of the calibration marker 21. Therefore, the first straight line 23 and the second straight line 24 are perpendicular to each other.

[0051] Moreover, the calibration marker 21 is formed of a first primary color, and the calibration marker 21 is further provided with a pattern made of a color not including the first primary color. A peripheral portion 22, which is a portion of the calibration chart 20 other than the calibration marker 21, is also provided with a pattern made of a color not including the first primary color. In FIG. 4, the first primary color is represented by diagonal lines, and a color not including the first primary color is represented by dots. For example, the first primary color may be blue, and the color not including the first primary color may be yellow. In this case, the calibration marker 21 is formed of blue, and a yellow pattern is provided. In the following, the explanation will be given mainly assuming an example in which the first primary color is blue, and the color not including the first primary color is yellow, but the first primary color and the color not including the first primary color may be other colors.

[0052] The pattern is a collection of irregularly arranged pattern elements. In the example shown in Figure 4, the pattern is formed by a collection of irregularly arranged dots. The pattern may also be a mottled pattern.

[0053] The marker detection unit 151 extracts a red component as the second primary color from an RGB image including such a calibration chart 20. The processed image obtained by this extraction is shown in FIG.

[0054] 5 is an explanatory diagram showing a specific example of a processed image made up of the red component of an RGB image. The yellow color of the pattern contains green and red components. Therefore, in the processed image, the green component of the pattern is lost, making the pattern fade. In addition, the blue color of the calibration markers 21 does not contain a red component, so in the processed image it has no luminance and appears black.

[0055] The marker detection unit 151 detects the positions of the two calibration markers 21 from such a processed image. Specifically, the marker detection unit 151 may detect the positions of the four corner points of each calibration marker 21. Note that since the pattern remains in the processed image, although it has become faded, the marker detection unit 151 may remove the pattern by, for example, binarizing the processed image. The pattern may also be formed in a second primary color, for example, red, and in this case, the pattern has already been removed in the processed image consisting of the red component.

[0056] (Depth information acquisition unit 152) The depth information acquisition unit 152 acquires depth information of a planar position in the distance image that corresponds to the position of the calibration marker 21 detected by the marker detection unit 151. For example, the depth information acquisition unit 152 acquires depth information of a planar position that corresponds to the center position of the calibration chart 20 specified based on the position of the calibration marker 21. When the imaging range and size of the RGB image and the distance image are equal, the depth information acquisition unit 152 acquires depth information of a planar position in the distance image that is equal to the center position of the calibration chart 20 in the RGB image. Here, it is assumed that the imaging range of the RGB image and the imaging range of the distance image are equal.

[0057] (Measurement range determination unit 153) The measurement range determination unit 153 determines the measurement range H based on the planar position in the distance image and the depth information at the planar position acquired by the depth information acquisition unit 152. The measurement range H may be, for example, a cube with 600 mm sides. In this case, the measurement range determination unit 153 may determine as the measurement range H a range in which the values ​​on two horizontal axes from the center position P of the calibration chart 20 shown in FIG. 5 are −300 mm to 300 mm, and the values ​​in the height direction from the center position P are 0 to 600 mm.

[0058] (Storage unit 154) The storage unit 154 stores various information used in the operation of the automatic reception machine 10. For example, the storage unit 154 stores the detection result of the position of the calibration marker 21 by the marker detection unit 151, the acquisition result of the depth information by the depth information acquisition unit 152, and information indicating the measurement range H determined by the measurement range determination unit 153.

[0059] (Size measurement section 155) When calibration is completed and operation of the automatic reception machine 10 is started, the size measurement unit 155 measures the size of the baggage placed on the placement surface F using the distance image acquired by the imaging unit 140. Specifically, the size measurement unit 155 generates point cloud data from the distance image acquired by the imaging unit 140, and converts the coordinate system of the point cloud data from the camera coordinate system to the coordinate system of the measurement range H. Furthermore, the size measurement unit 155 reads information indicating the measurement range H from the storage unit 154, extracts point cloud data within the measurement range H from the point cloud data, and measures the size of the baggage from the extracted point cloud data.

[0060] (Display control unit 156) The display control unit 156 generates various display screens and causes the generated display screens to be displayed on the operation display unit 110. For example, the display control unit 156 generates a measurement result screen that shows the result of measurement of the size of the luggage by the size measurement unit 155, and causes the measurement result screen to be displayed on the operation display unit 110.

[0061] <1-3. Operation> The functions of the control unit 150 according to the first embodiment of the present invention have been described above. Next, the calibration operation according to the first embodiment of the present invention will be summarized with reference to FIG.

[0062] 6 is a flowchart showing the operation of calibration according to the first embodiment of the present invention. First, a maintenance person places the calibration chart 20 on the placement surface F (S202). Then, the imaging unit 140 captures an image of the placement surface F to obtain an RGB image and a distance image (S204, S206).

[0063] Next, the marker detection unit 151 extracts the red component from the RGB image acquired in S204 to generate a processed image (S208). Then, the marker detection unit 151 detects the position of the calibration marker 21 from the processed image (S210).

[0064] Furthermore, the depth information acquisition unit 152 acquires depth information of a planar position corresponding to the position of the calibration marker 21 detected in S210, from among the planar positions in the range image acquired in S206 (S212).

[0065] Then, the measurement range determination unit 153 determines the measurement range H based on the planar position in the distance image and the depth information acquired in S212 (S214). Information indicating the determined measurement range H is stored in the storage unit 154.

[0066] <1-4. Effects> According to the first embodiment of the present invention described above, various operational effects can be obtained. For example, the calibration chart 20 according to the first embodiment of the present invention has a pattern in which pattern elements are irregularly arranged. Therefore, the difference information obtained from the two parallax images of the calibration chart 20 increases, so that more accurate depth information can be obtained. Furthermore, the calibration marker 21 is formed in, for example, blue as the first primary color, and the pattern is formed in, for example, yellow as a color not including the first primary color. According to this configuration, by extracting the component of the second primary color from the RGB image, the pattern becomes less noticeable compared to the calibration marker 21. Therefore, it is possible to detect the position of the calibration marker 21 while suppressing the adverse effect of the pattern. As a result, it is possible to more accurately determine the measurement range H expressed by the relative position from the imaging unit 140.

[0067] Furthermore, a first straight line 23 and a second straight line 24 that are perpendicular to each other are formed on the calibration chart 20. Furthermore, when an RGB image acquired by the imaging unit 140 during calibration is displayed as a live view on the operation display unit 110, a crosshair may be placed at the center of the live view display. In this case, a maintenance person can easily adjust the orientation of the imaging unit 140 by moving the imaging unit 140 so that the first straight line 23 and the second straight line 24 are aligned with the crosshair.

[0068] The automatic reception machine 10 may further include a weighing scale that measures the weight of the luggage, and the upper plate of the weighing scale may constitute the placement surface F. In this case, the calibration chart 20 may be configured to be magnetic. With this configuration, the calibration chart 20 is attached to the upper plate of the weighing scale by magnetic force, improving the flatness of the calibration chart 20. The shape and size of the calibration chart 20 may be the same as the shape and size of the upper plate of the weighing scale in a plan view. With this configuration, it is possible for a maintenance person to easily place the calibration chart 20 in the correct position (a position that fits the upper plate) without having to measure the position where the calibration chart 20 should be placed with a tape measure or the like.

[0069] The above-mentioned calibration may be started by a maintenance person operating a calibration menu screen displayed on the operation display unit 110. Alternatively, the above-mentioned calibration may be started automatically based on the detection of the calibration marker 21 of the calibration chart 20 by the marker detection unit 151. With this configuration, the calibration can be easily performed. Furthermore, as long as there is the calibration chart 20, the calibration can be performed even without the presence of a maintenance person.

[0070] <<Second embodiment>> The first embodiment of the present invention has been described above. As described above, according to the first embodiment of the present invention, it is possible to more accurately determine the measurement range H, which is expressed by the relative position from the imaging unit 140, during calibration. However, it is possible that the position of the imaging unit 140 may shift from the position of the imaging unit 140 during calibration after the operation of the automatic reception machine 10 is started. If the size of a package is measured with the imaging unit 140 in a shifted position, an inaccurate range is applied as the measurement range H, which causes a problem that the measurement result is also inaccurate. This problem can be solved by performing calibration to determine the measurement range H every time a measurement is made, but it is not realistic in terms of workload and time consumption for a maintenance worker to perform the calibration every time a measurement is made.

[0071] Taking the above circumstances into consideration, the present inventor has come up with the creation of a second embodiment of the present invention. According to the second embodiment of the present invention, it is possible to determine whether or not a positional deviation of the imaging unit 140 has occurred after the calibration that determines the measurement range H. The second embodiment of the present invention will be described below.

[0072] <2-1. External appearance of the automatic reception machine> Fig. 7 is an explanatory diagram showing the external configuration of the automatic reception machine 12 according to the second embodiment. As shown in Fig. 7, the automatic reception machine 12 according to the second embodiment of the present invention has an operation display unit 110, a receipt printer 120, a support unit 130, an imaging unit 140, and a control unit 160. The configurations of the operation display unit 110, the receipt printer 120, the support unit 130, and the imaging unit 140 are the same as those described in the first embodiment.

[0073] Four misalignment detection markers 32 are formed on the placement surface F. For clarity of the drawing, the misalignment detection markers 32 are shown with diagonal lines in Fig. 7, but in reality, the misalignment detection markers 32 may be two-dimensional codes. Fig. 4 shows an example in which the misalignment detection markers 32 are formed at each vertex position of a rectangle on the placement surface F, but the positions of the misalignment detection markers 32 are not limited to this example. In addition, the number of misalignment detection markers 32 is not limited to four, and may be three or less, or five or more.

[0074] The control unit 160 controls the overall operation of the automatic reception machine 12. In particular, the control unit 160 according to the second embodiment has a function of determining positional deviation of the imaging unit 140. The function of the control unit 160 according to the second embodiment will be described in detail below.

[0075] <2-2. Functions of the control unit> FIG. 8 is an explanatory diagram showing the function of the control unit 160 according to the second embodiment of the present invention. As shown in FIG. 8, the control unit 160 according to the second embodiment of the present invention has a measurement range determination unit 153, a size measurement unit 155, a display control unit 156, a marker detection unit 161, a depth information acquisition unit 162, a storage unit 164, and a comparison unit 167. The measurement range determination unit 153, the size measurement unit 155, and the display control unit 156 are as described in the first embodiment, so detailed description here will be omitted. The marker detection unit 161, the depth information acquisition unit 162, and the storage unit 164 have additional functions to the marker detection unit 151, the depth information acquisition unit 152, and the storage unit 154 described in the first embodiment in order to determine the positional deviation of the imaging unit 140.

[0076] (marker detection unit 161) The marker detection unit 161 is an example of an acquisition unit that detects the misalignment detection markers 32 from the RGB image acquired by the imaging unit 140 when the calibration described in the first embodiment is performed (first time point) and thereby acquires first position information indicating the positional relationship between the imaging unit 140 and the placement surface F. For example, the marker detection unit 161 may detect the positions of the four corner points of each of the misalignment detection markers 32 as the first position information, or may detect the center position of the four corner points of each of the misalignment detection markers 32.

[0077] In addition, each time the requester uses the automatic reception machine 12 after the calibration is performed (second time point), the marker detection unit 161 detects the deviation detection marker 32 from the RGB image acquired by the imaging unit 140 before the luggage is placed on the loading surface F, thereby acquiring first position information indicating the positional relationship between the imaging unit 140 and the loading surface F.

[0078] (Depth information acquisition unit 162) The depth information acquisition unit 162 is an example of an acquisition unit that acquires, as second position information, depth information of planar positions corresponding to positions indicated by the first position information in the distance image when performing the calibration described in the first embodiment. For example, when the first position information indicates positions of the four corner points of each deviation detection marker 32, the depth information acquisition unit 162 acquires depth information of each planar position corresponding to the four corner points of each deviation detection marker 32 in the distance image. When the first position information indicates a center position of the four corner points of each deviation detection marker 32, the depth information acquisition unit 162 acquires depth information of a planar position corresponding to the center position in the distance image. Note that when the imaging range and size of the RGB image and the distance image are equal, the positions of the four corner points of each deviation detection marker 32 in the RGB image coincide with each planar position corresponding to the four corner points of each deviation detection marker 32 in the distance image.

[0079] In addition, the depth information acquisition unit 162 acquires depth information of a planar position corresponding to the position indicated by the first position information detected by the marker detection unit 161 from the distance image each time the requester uses the automatic reception machine 12.

[0080] (Storage unit 164) The storage unit 164 stores various information used in the operation of the automatic reception machine 12. In particular, the storage unit 164 according to the second embodiment stores, as reference position information, position information acquired by the marker detection unit 161 and the depth information acquisition unit 162 when calibration is performed, in addition to the information stored in the storage unit 154 according to the first embodiment.

[0081] (Comparison section 167) The comparison unit 167 compares the reference position information stored in the storage unit 164 with the position information acquired at that time each time the requester uses the automatic reception machine 12. When a predetermined comparison result is obtained, the size measurement unit 155 executes the measurement of the baggage placed on the placement surface F by the requester. The predetermined comparison result may be a comparison result indicating that the difference between the reference position information stored in the storage unit 164 and the position information acquired at that time by the marker detection unit 161 and the depth information acquisition unit 162 is equal to or less than a threshold. When the difference is equal to or less than a threshold, it is considered that no positional deviation of the imaging unit 140 has occurred. In other words, the above comparison can be said to be a judgment of whether or not a positional deviation of the imaging unit 140 has occurred.

[0082] On the other hand, if a comparison result is obtained indicating that the difference exceeds the threshold, it is considered that a positional deviation of the imaging unit 140 has occurred. Therefore, the size measurement unit 155 does not measure the baggage placed on the placement surface F by the requester. In this case, the display control unit 156 may further generate an adjustment guidance screen that guides the adjustment of the position or attitude of the imaging unit 140. Then, when the maintenance person arrives in front of the automatic reception machine 12, the maintenance person starts adjusting the position or attitude of the imaging unit 140.

[0083] In addition, on the adjustment guidance screen, an arrow indicating the direction in which the imaging unit 140 should be tilted may be displayed, and the size of the angle to be tilted may be expressed by the size of the arrow. While the deviation in the position or posture of the imaging unit 140 is not eliminated, a message to that effect, such as NG, is displayed on the adjustment guidance screen, and when the deviation in position is eliminated, a message to that effect, such as OK, is displayed on the adjustment guidance screen. After that, the automatic reception machine 12 can be used.

[0084] <2-3. Operation> The functions of the control unit 160 according to the second embodiment of the present invention have been described above. Next, the operation during calibration according to the second embodiment will be summarized with reference to Fig. 9, and the operation during use of the automatic reception machine 12 will be summarized with reference to Fig. 10.

[0085] (Calibration operation) 9 is a flowchart showing the operation during calibration according to the second embodiment. The processes of S202 to S214 are as described in the first embodiment. In the second embodiment, the processes of S222 to S226 are added.

[0086] Specifically, the marker detection unit 161 detects the misalignment detection markers 32 from the RGB image acquired by the imaging unit 140 to acquire first position information indicating the positional relationship between the imaging unit 140 and the placement surface F (S222). Then, the depth information acquisition unit 162 acquires depth information of a planar position corresponding to the position indicated by the first position information in the distance image as second position information (S224). Furthermore, the storage unit 164 stores the position information acquired by the marker detection unit 161 and the depth information acquisition unit 162 as reference position information (S226).

[0087] (Operation when in use) 10 is a flow chart showing the operation of the automatic reception machine 12 according to the second embodiment when in use. The process shown in FIG. 10 is started after S71 shown in FIG.

[0088] First, the imaging unit 140 acquires an RGB image and a distance image of the placement surface F before a baggage is placed on the placement surface F (S302, S304). Then, the marker detection unit 161 detects the deviation detection marker 32 from the RGB image acquired by the imaging unit 140 to acquire first position information indicating the positional relationship between the imaging unit 140 and the placement surface F (S306). In addition, the depth information acquisition unit 162 acquires, as second position information, depth information of a planar position corresponding to the position indicated by the first position information in the distance image (S308).

[0089] Then, the comparison unit 167 compares the reference position information stored in the storage unit 164 with the position information acquired in S306 and S308 (S310). If the difference between the reference position information stored in the storage unit 164 and the position information acquired in S306 and S308 is equal to or less than the threshold (S310 / No), it is considered that no positional deviation of the imaging unit 140 has occurred, and the automatic reception machine 12 executes the process from S72 onwards shown in FIG. 2.

[0090] On the other hand, if the difference is greater than or equal to the threshold value (S310 / Yes), it is assumed that a position shift has occurred in the imaging unit 140, so the automatic reception machine 12 does not measure the baggage and displays an adjustment guidance screen on the operation display unit 110 (S312).

[0091] <2-4. Effects> According to the second embodiment of the present invention described above, various operational effects can be obtained. For example, the control unit 160 according to the second embodiment of the present invention acquires position information indicating the positional relationship between the imaging unit 140 and the placement surface F when performing calibration for determining the measurement range H, and stores the information as reference position information. Then, each time the client uses the device, the control unit 160 can determine whether or not the imaging unit 140 has shifted in position by comparing the position information acquired at that time with the reference position information.

[0092] Here, the position information includes first position information indicating a position according to the detection result of the marker in the visible light image, and thus by comparing such position information with the reference position information, it is possible to determine whether or not there is a positional shift of the imaging unit 140 in a direction parallel to the placement surface F and whether or not there is a change in the posture of the imaging unit 140. In addition, the position information includes, as second position information, depth information of a planar position corresponding to the position indicated by the first position information in the distance image. By comparing such position information with the reference position information, it is possible to determine whether or not there is a positional shift of the imaging unit 140 in a direction perpendicular to the placement surface F.

[0093] According to the second embodiment of the present invention, if the difference between the location information acquired during use by the client and the reference location information is equal to or less than a threshold, the measurement of the package is performed. On the other hand, if a comparison result is obtained indicating that the difference exceeds the threshold, the measurement of the package is not performed. This configuration can prevent erroneous measurement results from being obtained.

[0094] Furthermore, according to the second embodiment of the present invention, when a comparison result indicating that the difference exceeds a threshold is obtained, the display control unit 156 generates an adjustment guidance screen that guides the maintenance person to adjust the position or attitude of the imaging unit 140. With this configuration, the maintenance person can easily adjust the imaging unit 140 while viewing the adjustment guidance screen.

[0095] <<3. Modifications>> An embodiment of the present invention has been described above. In the following, some modified examples of the above-described embodiment will be described. Note that each modified example described below may be applied alone to the above-described embodiment, or may be applied in combination to the above-described embodiment. Also, each modified example may be applied in place of the configuration of the above-described embodiment, or may be applied in addition to the configuration of the above-described embodiment.

[0096] In the above, an example has been described in which the counter top is used as the placement surface F, but the counter top may be movable or removable, and the placement surface F on which the misalignment detection marker 32 is formed may appear by moving or removing the counter top. In this case, the control unit 160 according to the second embodiment may automatically start processing for measuring the luggage based on the fact that the misalignment detection marker 32 has begun to be detected by the marker detection unit 161. Alternatively, the control unit 160 according to the second embodiment may automatically start processing for measuring the luggage based on the fact that the movement or removal of the counter top has been detected.

[0097] As a modification of the second embodiment, the imaging unit 140 may have a built-in angle sensor such as a 6DoF sensor. In this case, the detection result obtained by the angle sensor when the calibration is performed is stored in the storage unit 164 as reference information, and the presence or absence of a deviation in the posture of the imaging unit 140 can be determined by comparing the detection result obtained by the angle sensor with the reference information when the client uses the imaging unit 140.

[0098] In the first embodiment, the calibration chart 20 is used to perform the calibration to determine the measurement range H. However, the measurement range H can be determined using the deviation detection marker 32 described in the second embodiment. For example, the measurement range determination unit 153 may determine the measurement surface and the measurement range H based on the detection position of the deviation detection marker 32. Specifically, the measurement range determination unit 153 may determine a square or rectangle circumscribing the multiple deviation detection markers 32 as the bottom surface of the measurement range H. With this configuration, in counters with different widths, the size of the measurement range H can be easily changed by varying the distance between the multiple deviation detection markers 32. In addition, even after the automatic reception machine 12 has started operating, the measurement range H can be easily moved by changing the position of the deviation detection marker 32 and performing the calibration to determine the measurement range H again.

[0099] <<Additional Information>> Although the preferred embodiment of the present invention has been described in detail above with reference to the accompanying drawings, the present invention is not limited to such an example. It is clear that a person having ordinary knowledge in the technical field to which the present invention pertains can conceive of various modified or altered examples within the scope of the technical ideas described in the claims, and it is understood that these also naturally belong to the technical scope of the present invention.

[0100] For example, the steps in the processing of the automatic reception machine 10 and the automatic reception machine 12 in this specification do not necessarily have to be processed in chronological order according to the order described in the sequence diagram or flowchart. For example, the steps in the processing of the automatic reception machine 10 and the automatic reception machine 12 may be processed in an order different from the order described in the flowchart, or may be processed in parallel.

[0101] It is also possible to create a computer program for causing hardware such as the CPU, ROM, and RAM built into the automatic reception machine 10 and the automatic reception machine 12 to perform functions equivalent to those of the above-mentioned components of the automatic reception machine 10 and the automatic reception machine 12. A non-transitory storage medium storing the computer program is also provided. [Explanation of symbols]

[0102] 10, 12 Automatic reception machine 110 Operation display section 120 Receipt Printer 130 Support part 140 Imaging unit 150, 160 control unit 151, 161 Marker detection unit 152, 162 Depth information acquisition unit 153 Measurement range determination unit 154, 164 Storage section 155 size family bottom 156 Display control section 167 Comparison Section

Claims

1. an imaging unit that captures an image of a target surface including the marker; an acquisition unit that acquires position information indicating a positional relationship between the imaging unit and the target surface based on detection of the marker from the captured image; A storage unit that stores reference position information; a comparison unit that compares the location information acquired by the acquisition unit with the reference location information stored in the storage unit; Equipped with the captured image acquired by the imaging unit includes a visible light image and a distance image, a marker processing device, wherein the position information includes first position information indicating a position according to a detection result of the marker in the visible light image, and second position information indicating depth information of a planar position corresponding to the position indicated by the first position information in the distance image.

2. the reference position information is position information acquired based on detection of the marker from a first captured image acquired by imaging at a first time point, The marker processing device according to claim 1 , wherein the comparison unit compares the position information acquired based on detection of a marker from a second captured image acquired by imaging at a second time point later than the first time point with the reference position information.

3. The marker processing device further includes a size measurement unit that measures a size of an object based on the distance image acquired by the imaging unit in a state where the object is placed on the target surface, The marker processing device according to claim 1 , wherein the size measurement unit measures the size of the object based on a predetermined comparison result obtained by the comparison unit.

4. The marker processing device according to claim 3 , wherein the predetermined comparison result is a comparison result indicating that a difference between the position information acquired by the acquisition unit and the reference position information is equal to or smaller than a threshold value.

5. The marker processing device according to claim 4 , wherein, when the comparison section obtains a comparison result indicating that the difference exceeds the threshold value, the size measurement section does not perform measurement of the size of the object.

6. The marker processing device according to claim 3 , further comprising a display control unit that generates a measurement result screen that shows a result of the measurement by the size measurement unit.

7. The marker processing device according to claim 5, further comprising a display control unit that generates an adjustment guidance screen that guides the user to adjust a position or attitude of the imaging unit when the comparison unit obtains a comparison result indicating that the difference exceeds the threshold value.

8. a marker formed of a first primary color; The marker is provided with a pattern made of a color that does not include the first primary color, A marker sheet, wherein the pattern is a collection of randomly arranged pattern elements.

9. A marker formed of a first primary color, The marker is provided with a pattern made of a color that does not include the first primary color. A magnetic marker sheet.

10. A marker sheet comprising: a marker formed of a first primary color; The marker is provided with a pattern made of a color that does not include the first primary color, the marker has an outer shape that is a rectangle having a first side and a second side that is perpendicular to the first side; The marker sheet has a first straight line parallel to the first side and a second straight line parallel to the second side formed so as to intersect with each other.

11. an imaging unit that captures an image of a marker sheet that is placed on a target surface and includes markers that are formed in a first primary color and have a pattern made of a color that does not include the first primary color, to obtain a visible light image and a distance image; a detection unit that detects a position of the marker in the visible light image; an acquisition unit that acquires depth information of a planar position in the distance image corresponding to a detection result by the detection unit; a measurement range determination unit that determines a measurement range based on the planar position and the depth information; a size measurement unit that measures a size of an object by extracting data within the measurement range from the distance image acquired by the imaging unit while the object is placed on the target surface; Equipped with The detection unit detects the position of the marker from a processed image obtained by extracting a second primary color component from the visible light image.

Images