Determination device, determination method, and program

Abstract

To provide a determination device for giving notice to a user when there is no article which should be present in a predetermined area, and to provide a determination method and a program therefor.SOLUTION: A determination system 1000 comprises a determination device 100, an imaging device 200, and a projection device 300. The imaging device 200 images a predetermined area where an article (object) which should be present is originally placed. The determination device 100 determines whether or not the object is present in a place where it is originally placed, based on an image imaged by the imaging device 200. The projection device 300 projects warning to the place where the object is originally placed, when the object is not present in the place where it is originally placed, in order to give notice of the foregoing.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a determination device, a determination method, and a program. [Background technology]

[0002] Projection AR (Augmented Reality) technology is known, which uses a projector to project information onto real-world objects by superimposing the information. As an application example of the technology that uses a projector to project information onto real-world objects, for example, Patent Document 1 discloses a projection system that notifies the owner of a piece of luggage that the owner has left the luggage in an easy-to-understand manner. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2015-23459 A Summary of the Invention [Problem to be solved by the invention]

[0004] The projection system disclosed in Patent Document 1 detects the positional relationship between the baggage and the baggage owner in a specified area, and when the distance between the owner and the baggage exceeds a threshold, a specified projection is performed to notify the owner that the baggage has been left behind in an easily understandable manner. However, this conventional technology has a problem in that, although it can notify the user that the baggage has been left behind, it cannot notify the user when an item that should be there is missing in the specified area.

[0005] The present invention has been made in consideration of the above-mentioned situation, and aims to provide a determination device, a determination method, and a program that can notify a user when an object that should be there is missing in a specified area. [Means for solving the problem]

[0006] In order to achieve the above object, one aspect of the determination device according to the present invention is to A captured image is obtained by capturing an image of a predetermined area; determining whether a predetermined object is present in the captured image; When it is determined that the predetermined object does not exist, notifying that the object does not exist. It has a control unit. Effect of the Invention

[0007] According to the present invention, when an object that should be there is missing in a predetermined area, a user can be notified. [Brief description of the drawings]

[0008] [Figure 1] FIG. 2 is a diagram illustrating an example of a functional configuration of a determination system according to an embodiment. [Diagram 2] FIG. 2 is a diagram showing an example of connections between devices constituting a determination system according to an embodiment. [Diagram 3] 13 is a flowchart illustrating an example of an initial setting process according to the embodiment. [Figure 4] FIG. 13 is a diagram showing an example of a determination image. [Diagram 5] 13 is a diagram showing an example in which a dotted rectangle is displayed around an object recognized in the determination image. FIG. [Figure 6] FIG. 11 is a diagram showing an example of determination data according to the embodiment; [Figure 7] 13 is a flowchart illustrating an example of a determination process according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] The determination system and the like according to the embodiment will be described with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals.

[0010] (Embodiment) As shown in FIG. 1, the determination system 1000 according to the embodiment includes a determination device 100, a photographing device 200, a projection device 300, and a rotation device 400. The photographing device 200 photographs a predetermined area where an object (target) is originally placed, for example, a storage area for cooking utensils, tools, and other utensils. The rotation device 400 is a turntable that rotates the photographing device 200 and the projection device 300, and is a device for expanding the area that can be photographed by the photographing device 200 (and the area that can be projected by the projection device 300). The photographing direction (area that can be photographed) of the photographing device 200 and the projection direction (area that can be projected) of the projection device 300 can be changed by rotating the rotation device 400. If the predetermined area such as a storage area for utensils is a narrow area (if all the utensils such as cooking utensils and tools fit within the angle of view of the camera lens of the photographing device 200), the rotation device 400 is not necessary.

[0011] The determination device 100 determines whether or not the object is present at the location where it should be placed, based on the image captured by the imaging device 200. If the object is not present at the location where it should be placed, the projection device 300 projects a warning (e.g., a red rectangle) at the location (where the object is originally placed) to notify the user of the absence.

[0012] 2, the determination system 1000 is configured by connecting a determination device 100 to an imaging device 200, a projection device 300, and a rotation device 400. These connections may be wired (e.g., a Universal Serial Bus (USB) cable, a High-Definition Multimedia Interface (HDMI) (registered trademark), etc.) or wireless (e.g., a wireless Local Area Network (LAN), etc.).

[0013] The determination device 100 is a PC (Personal Computer), and includes, as its functional configuration, a control unit 110, a storage unit 120, an operation input unit 130, and a display unit 140, as shown in FIG.

[0014] The control unit 110 is composed of a processor such as a CPU (Central Processing Unit). The control unit 110 executes processes for implementing various functions of the determination system 1000 and a determination process (to be described later) using a program stored in the storage unit 120. The control unit 110 also has a timer function and can measure time. The control unit 110 also supports multi-threading and can execute multiple processes in parallel.

[0015] The storage unit 120 stores programs executed by the control unit 110 and necessary data. The storage unit 120 may include, but is not limited to, a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, etc. Note that the storage unit 120 may be provided inside the control unit 110.

[0016] The operation input unit 130 is a user interface such as a keyboard, a mouse, etc., and receives operation input from a user. When the operation input unit 130 includes a touch panel, the touch panel may be integrated with the display of the display unit 140.

[0017] Display unit 140 includes a display such as a liquid crystal display or an organic EL (Electro-Luminescence) display, and displays information to be presented to the user.

[0018] Although not shown, the determination device 100 may include a communication unit such as a wireless LAN interface and be capable of communicating with an external device (such as a server or a smartphone). In that case, for example, the determination device 100 may transmit the determination result to another device (for example, by email to the user's smartphone) via the communication unit.

[0019] The photographing device 200 is a camera that photographs a predetermined area and transfers data of the photographed image (photographed image) to the determination device 100. Therefore, the control unit 110 can obtain the photographed image of the predetermined area photographed by the photographing device 200.

[0020] The projection device 300 is a projector, and acquires an image designated (generated) by the determination device 100 via a communication unit (not shown), and projects the acquired image. Therefore, the control unit 110 can make the projection device 300 project a predetermined image. Usually, a dedicated screen is prepared for the projector, and an image is generally projected onto the screen, but in the present embodiment, a dedicated screen is not necessary, and the projection device 300 projects directly onto a normal wall or the like. Here, the normal wall or the like includes a wall having irregularities and a cloth having irregularities such as a curtain. The projection device 300 as a whole is also called a projection unit. The projection unit inside the projection device 300 refers to components including a light source unit (not shown), an optical unit such as a lens or mirror that guides the light emitted from the light source unit, and a display unit such as a DMD (Digital Micromirror Device) that collects the light guided by the optical unit.

[0021] A calibration process is performed when the determination system 1000 is installed so that the image projected by the projection device 300 can be correctly projected onto a surface such as a normal wall. This calibration process is a process for establishing a correspondence between coordinates in the image projected by the projection device 300 (projector coordinates) and coordinates in the image captured by the image capture device 200 (camera coordinates).

[0022] The calibration process may be performed in any manner, but for example, a checkerboard pattern image is first projected by the projection device 300, and the projected image is then photographed by the photographing device 200. The control unit 110 of the determination device 100 then obtains shape correction information from the correspondence between the coordinates (camera coordinates) of each intersection in the photographed check image and the coordinates (projector coordinates) of each intersection in the original check image. By using this shape correction information, the control unit 110 knows what kind of image (image expressed in projector coordinates) should be projected in order to project an image of a specific shape onto, for example, a location (area expressed in camera coordinates) where an object should have been.

[0023] It should be noted that if the position of the camera lens of the image capturing device 200 and the position of the projection lens of the projection device 300 are changed, it is necessary to perform the calibration process again. Therefore, the image capturing device 200 may be fixed to the projection device 300 using a predetermined jig or the like, or the function of the image capturing device 200 may be incorporated into the projection device 300. In this way, the positional relationship between the camera lens and the projection lens does not change, and re-calibration processing based on a change in the positions of both lenses is not required, so this configuration is more preferable.

[0024] Furthermore, the control unit 110 may calculate color correction information based on not only the shape correction information but also the color correction information based on the color information of the image captured by the imaging device 200. By using the shape correction information and the color correction information, even if there is no dedicated screen, the projection device 300 can project a desired image onto a predetermined area by acquiring an image generated by the control unit 110 of the determination device 100 based on the shape correction information and the color correction information.

[0025] The rotation device 400 is a turntable, and rotates the projection device 300 and the image capturing device 200 by a specified angle. The rotation angle is specified by the control unit 110 of the determination device 100. When objects existing at multiple locations are to be determined (when objects exist in multiple different areas (projection areas, camera angle of view) in the projection direction of the projection device 300 and the image capturing direction of the image capturing device 200), the user stores angles at which they can be captured in advance in the storage unit 120 as an image capturing angle list (for example, array variables a[1] to a[n]).

[0026] In this embodiment, when the user installs the determination system 1000, the user installs the imaging device 200, the projection device 300, and the rotation device 400 at positions where the target object can be photographed, and performs the initial setting process shown in Fig. 3 to perform the initial setting. This process rotates the rotation device 400 to each imaging angle included in the imaging angle list, and performs the above-mentioned calibration process and imaging at the imaging angle, thereby grasping the location where each target object should be.

[0027] The initial setting process will be described with reference to Fig. 3. First, the control unit 110 of the determination device 100 acquires a shooting angle list (array variables a[1] to a[n]. Here, it is assumed that n shooting angles are set) (step S101). The shooting angle list is a list in which angles for rotating the rotation device 400 are set, and may be stored in advance in the storage unit 120 by the user, or in step S101, the user may rotate the rotation device 400 to set shooting angles at which each object can be photographed, and the control unit 110 may acquire the set shooting angles.

[0028] Next, the control unit 110 initializes the value of a variable i to 1 (step S102). Then, the control unit 110 rotates the rotation device 400 by an angle represented by the array variable a[i] (step S103).

[0029] Then, the control unit 110 executes the above-mentioned calibration process (step S104) to obtain shape correction information and the like at the shooting angle of a[i]. The control unit 110 may store the obtained shape correction information (which may include color correction information) in the storage unit 120 in association with each element (angle) of the array variable a[i].

[0030] Next, the control unit 110 captures an image (determination image) at the shooting angle of a[i] by the photographing device 200 (step S105). In this step, it is possible to simply capture image data captured by the photographing device 200 as the determination image, but in this embodiment, the control unit 110 sets the coordinates of the location where the object should be in the photographed image.

[0031] Specifically, for example, when the rotation angle of the rotation device 400 is 0 degrees, a predetermined area in which a bread knife 211, a Santoku knife 212, and a vegetable knife 213 are placed is photographed by the photographing device 200 to obtain a judgment image 210, as shown in Fig. 4. Then, the control unit 110 recognizes an object that is thought to be the target from within the judgment image 210, and displays it on the display unit 140, surrounded by dotted lines 216, 217, and 218, as shown in Fig. 5.

[0032] If the user determines that the recognition result displayed on display unit 140 is correct (the object surrounded by the dotted rectangle is the object the user wants to determine), the user can click or touch the dotted area with the mouse to set the coordinates surrounded by the dotted line as the coordinates of the location where the object should be.

[0033] For example, since all the recognition results shown in Fig. 5 are correct, the user touches each of the dotted lines 216, 217, and 218. As a result, as shown in Fig. 6, a judgment image 210 and the coordinates of each object (object coordinates 219) when the shooting angle is 0 degrees are stored in the storage unit 120 as judgment data 121. Note that, although image data captured by the imaging device 200 is directly used as the judgment image 210 in Fig. 6, images of each object may be cut out based on information on the object coordinates, etc., and these may be used as judgment images for each object. In this case, the storage unit 120 stores judgment images for each shooting angle, the number of which is the number of objects.

[0034] Returning to FIG. 3, the control unit 110 then adds 1 to the variable i (step S106) and determines whether the value of the variable i is greater than n (the number of elements in the array a of the shooting angle list) (step S107). If the value of the variable i is less than or equal to n (step S107; No), the process returns to step S103. As a result, the control unit 110 rotates the rotation device 400 to the next shooting angle, and then performs the above-mentioned calibration process and acquisition of a judgment image.

[0035] If the value of the variable i is greater than n (Step S107; Yes), the control unit 110 ends the initial setting process.

[0036] By this initial setting process, as shown in Fig. 6, judgment images 210, 220 and object coordinates 219, 229 at each shooting angle are acquired and stored in storage unit 120 as judgment data 121. In the example of judgment data 121 shown in Fig. 6, it can be seen that a bread knife 211, a Santoku knife 212, and a vegetable knife 213 exist at the coordinates indicated by object coordinates 219 of judgment image 210 in the region where the shooting angle is 0 degrees, and a large screwdriver 221 and a small screwdriver 222 exist at the coordinates indicated by object coordinates 229 of judgment image 220 in the region where the shooting angle is 180 degrees.

[0037] Next, a determination process in which the determination device 100 periodically (for example, early in the morning and after work) determines whether or not an object is present will be described with reference to Fig. 7. This process starts when the initial setting process ends.

[0038] First, the control unit 110 of the determination device 100 acquires a shooting angle list (array variables a[1] to a[n]) (step S201). This shooting angle list is the same as the one acquired in step S101 of the initial setting process (FIG. 3).

[0039] Then, the control unit 110 acquires a determination time list (array variables t[1] to t[m]; here, it is assumed that m determination times are set) (step S202). The determination time list is a list that sets the time at which to determine the presence or absence of an object, and may be stored in the storage unit 120 in advance by the user, or may be set by the user in step S202. In addition, two values ​​may be set as default settings: early morning (e.g., t[1]=7:00) and after work (e.g., t[2]=22:00) (in this example, m=2).

[0040] Next, the control unit 110 initializes the value of the variable j to 1 (step S203). Then, the control unit 110 judges whether the current time has become the time represented by the array variable t[j] (step S204). If the current time has not yet become the time represented by t[j] (step S204; No), the control unit 110 returns to step S204 and waits until the current time becomes the time represented by t[j].

[0041] When the current time reaches the time represented by t[j] (step S204; Yes), the control unit 110 initializes the value of the variable i to 1 (step S205). Then, the control unit 110 rotates the rotation device 400 by the angle represented by the array variable a[i] (step S206).

[0042] Next, the control unit 110 captures an image (confirmation image) at the imaging angle of a[i] by the imaging device 200 (step S207).

[0043] Then, the control unit 110 refers to the determination data 121, compares the determination image at the shooting angle a[i] with the confirmation image acquired in step S207, and determines whether or not all objects are present in the confirmation image (step S208). In this determination, the control unit 110 may simply compare two images (the determination image and the confirmation image) to make the determination, or may recognize objects contained in each image and compare the recognized objects to make the determination.

[0044] For example, when making a judgment by simply comparing images, the control unit 110 judges that all objects are present if there are no areas where the difference is greater than the threshold, and judges that no object corresponding to the object coordinates stored in the judgment data 121 is present if an area where the difference is greater than the threshold is included in the object coordinates stored in the judgment data 121.

[0045] In addition, when recognizing objects contained in an image and comparing the recognized objects to make a judgment, for each area of ​​object coordinates stored in judgment data 121, the objects in the judgment image and the objects in the confirmation image are recognized, and if the same object is recognized in the judgment image and the confirmation image for all object coordinates, it is judged that all objects are present, and if there are object coordinates in the confirmation image where no object is recognized, it is judged that no object corresponding to that object coordinate does not exist.

[0046] If all objects are present (step S208; Yes), the process proceeds to step S210. If any object is not present (step S208; No), the control unit 110 notifies an abnormality (the object is not present where it should be) (step S209).

[0047] Any method of notifying the abnormality may be used. For example, the control unit 110 may alternately project red and white rectangles at regular intervals using the projection device 300 into the area of ​​the object coordinates of the object determined not to exist, thereby notifying the user that the object does not exist where it should be. The confirmation image (and the judgment image, if necessary) may be sent to the user as an email attachment or the like. If the notification is sent by email, the user can confirm that the object does not exist in the area indicated by the object coordinates by looking at the confirmation image attached to the email wherever he or she is. Alternatively, instead of the red and white rectangles, the projection device 300 may project an image showing the object where it should be (for example, an image showing a bread knife 211) into the area where it should be. Here, the image showing the object where it should be may be generated from the judgment image 210 or the like in the judgment data 121.

[0048] When there are multiple objects that are not in the correct location at one shooting angle, the control unit 110 performs the above projection (alternately projecting red and white squares) on all of the object coordinate regions of each of the non-existent objects at once. After performing this projection for a predetermined time (e.g., one minute), the control unit 110 may proceed to the next process (the process after step S210), or may continue the projection without proceeding to the next process until the user inputs that an abnormality has been confirmed.

[0049] Furthermore, when a plurality of shooting angles are set in the shooting angle list, instead of notifying an abnormality each time in step S209, the shooting angle and object at which an abnormality was confirmed may be stored in the storage unit 120 as an abnormality detection list in step S209, and after the determination in step S211 becomes Yes, the abnormality may be notified based on the abnormality detection list stored in the storage unit 120 up to that point. In this way, it becomes possible to notify abnormalities collectively. Furthermore, when creating the determination data in the initial setting process, it is possible to set a priority for each object, and by doing so, it becomes possible to notify abnormalities in order of the object with the highest priority when notifying abnormalities collectively.

[0050] Returning to Fig. 7, next, the control unit 110 adds 1 to the variable i (step S210), and judges whether the value of the variable i is greater than n (the number of elements of the array a of the shooting angle list) (step S211). If the value of the variable i is less than or equal to n (step S211; No), the process returns to step S206. As a result, the control unit 110 rotates the rotation device 400 to the next shooting angle and then acquires a confirmation image to check whether or not the target object exists.

[0051] If the value of variable i is greater than n (step S211; Yes), the control unit 110 adds 1 to variable j (step S212) and determines whether the value of variable j is greater than m (the number of elements in array t of the determination time list) (step S213). If the value of variable j is equal to or less than m (step S213; No), the process returns to step S204. This causes the control unit 110 to wait until the next determination time and then check again for the presence or absence of an object from the initial shooting angle.

[0052] If the value of the variable j is greater than m (step S213; Yes), the process returns to step S203. This causes the control unit 110 to check again for the presence or absence of an object from the first determination time and the first shooting angle.

[0053] Through this determination process, if an object that should be present is not present in each area set in the shooting angle list at each time set in the determination time list, the user can be notified that the object is not present.

[0054] In particular, by setting early morning (before work begins) and closing time (after work is completed) in the judgment time list, it is possible to confirm that all tools, etc. are ready before work begins and after work is completed, and if an abnormality occurs, it becomes possible to immediately know when the abnormality occurred (whether it was in the middle of the night, during work today, etc.). Also, by adding work breaks (for example, 12:00 (start of lunch break), 1:00 (end of lunch break), 3:00 (break time), etc.) to the judgment time list, it becomes possible to know the time period in which the abnormality occurred in more detail.

[0055] By performing this determination process regularly and notifying the user whenever an abnormality occurs, the user can always keep things in order, things do not get lost, and accidents caused by missing items can be prevented.

[0056] (Variation 1) In the above-described embodiment, in step S208 of the determination process (FIG. 7), if all of the objects are not in the places where they should be and in the same orientation, the control unit 110 determines "No." However, there may be cases where objects are placed in their original locations with their orientation reversed, such as upside down, reversed left and right, or upside down. Modification 1 for dealing with such cases will be described.

[0057] In the first modification, in step S105 of the initial setting process (FIG. 3), the control unit 110 also acquires an image of each object with its orientation changed (for example, an image with its front and back reversed) as a determination image, and registers it in the determination data 121. Since image data of some objects is symmetrical left to right or top to bottom, in such a case, the control unit 110 can automatically generate an image of the object with its orientation changed from the image data in the determination image captured by the photographing device 200. In the case of an object with different images on the front and back, the control unit 110 asks the user to turn the object over, photographs it again with the photographing device 200, acquires an image of the object with its orientation changed, and registers it in the determination data 121.

[0058] Then, in step S208 of the judgment process (FIG. 7), control unit 110 recognizes the objects contained in the confirmation image, and for each object present in each region of the object coordinates of the judgment image, compares the object in the judgment image with the object in the confirmation image in all orientations registered in judgment data 121, and if there is a match in any orientation (the difference is less than a threshold value), it judges that the object is present.

[0059] In the above-described first modification, the control unit 110 can determine whether the object is present in the correct location, regardless of the orientation in which the user returns the object. Therefore, in the first modification, it is possible to prevent the object from being determined to be abnormal because the user placed the object in the wrong orientation, and the control unit 110 can correctly determine and notify the presence or absence of the object, regardless of the orientation in which the object is placed. Note that the orientation here is not limited to the orientation (direction) of the object, such as front / back, left / right, or up / down. The inclination of the object (the inclination with respect to the vertical or horizontal axis when the object placed on a surface is viewed from above (as in the determination image 210)) is also included in the orientation.

[0060] (Variation 2) In the above-described embodiment, in step S208 of the judgment process (FIG. 7), if all of the objects are not in the correct order in the places where they should be, the control unit 110 judges the result to be No. However, there may be cases where the order of placement is incorrect, such as placing the large screwdriver 221 and the small screwdriver 222 in reverse (for example, when placing the large screwdriver 221 and the small screwdriver 222 side by side, they are placed in the opposite order from the normal placement). Modification 2 for dealing with such cases will be described.

[0061] In variant example 2, in step S208 of the judgment process (FIG. 7), control unit 110 recognizes objects contained in the confirmation image, and for each object in the judgment image, even if the object coordinates are different, if the object images match (the difference is less than a threshold), it judges that the object is present.

[0062] However, since there may be multiple objects of the same shape, the control unit 110 also counts the number of objects determined to exist, and if the number of objects included in the confirmation image differs from the number of objects included in the determination image, it determines that one of the objects does not exist. In this case, since it is not possible to determine which of the objects does not exist, the control unit 110 may project a predetermined image (e.g., a red square and a white square) onto any of the locations where the object should be in step S209, or may project a predetermined image onto all of the locations where the object should be.

[0063] In the above-described modified example 2, even if an object is not in a place where it should be, control unit 110 can determine the presence or absence of the object without relying on the object coordinates. Therefore, modified example 2 can prevent an abnormality from being determined as occurring because the user has returned the object to the wrong place, and even if the order in which the objects are placed is incorrect, control unit 110 can correctly determine and notify the presence or absence of the object.

[0064] (Variation 3) In the above embodiment, in step S208 of the judgment process (FIG. 7), control unit 110 judges "No" if the shape of the object has changed. However, there may be cases where the shape of the same object changes, such as the extent to which the lead of a mechanical pencil is extended, or when scissors are opened or closed. Modification 3 for dealing with such cases will be described.

[0065] In variant example 3, in step S105 of the initial setting process (FIG. 3), control unit 110 also acquires images of the parts of each object whose shape does not change (unchanging parts) (for example, an image of the main body of a mechanical pencil excluding the tip, and images of each of the two parts of scissors connected at the fulcrum (an image of the part of the lower blade (the part of the handle where the thumb goes) and an image of the part of the upper blade (the part of the handle where the index finger goes)) as judgment images and registers them in judgment data 121. When registering, control unit 110 may display the judgment image on display unit 140 and have the user indicate which part should be registered as the unchanging part of the object by, for example, tracing the unchanging part of the object with a mouse.

[0066] Then, in step S208 of the judgment process (FIG. 7), the control unit 110 recognizes the objects contained in the confirmation image, and for each object present in each region of the object coordinates in the judgment image, compares the "parts whose shape does not change (unchanging parts)" registered in the judgment data 121 between the object in the judgment image and the object in the confirmation image, and if they match (the difference is less than a threshold value), it judges that the object is present.

[0067] In the above-described third variant, even if the shape of the object has changed when the user puts the object back, control unit 110 can determine the presence or absence of the object regardless of the change in shape. This prevents the user from judging an abnormality as occurring, for example, when the user puts the scissors back to their place with the scissors left open. Even if the shape of the object has changed, control unit 110 can correctly determine and notify the presence or absence of the object.

[0068] (Variation 4) In the above embodiment, in step S208 of the determination process (FIG. 7), if the color of the object has changed, the control unit 110 determines "No." However, for example, there may be cases where the color of the same object changes due to partial discoloration caused by aging. Modification 4 for dealing with such cases will be described.

[0069] In the fourth modification, in step S208 of the judgment process (FIG. 7), the control unit 110 recognizes objects contained in the confirmation image, and compares the objects in the judgment image and the confirmation image for each object present in each region of the object coordinates of the judgment image, and converts the RGB (Red Green Blue) data of the judgment image and the confirmation image into HSV (Hue Satuation Value) data for color, compares the HSV data, and judges that the object exists if the hue matches (the difference is less than a threshold value) even if the saturation and value do not match. Note that this judgment method is just one example, and the control unit 110 may use any judgment method in step S208 as long as it can ignore partial discoloration due to aging.

[0070] In the above-described fourth modification, even if the color of the object has changed, the control unit 110 can determine the presence or absence of the object without relying on the change in color. Therefore, in the fourth modification, it is possible to prevent the object from being determined to be abnormal because the color of the object has changed due to aging or the like, and the control unit 110 can correctly determine and notify the presence or absence of the object even if the color of the object has changed.

[0071] (Variation 5) In the above-described embodiment, notification of an abnormality is mainly performed by the projection device 300. However, for example, notification of an abnormality may be performed by the display unit 140, and the projection device 300 is not essential to the determination system 1000. However, the determination device 100 may be installed in a location other than the site (the location where the target object is placed), and in such a case, even if notification of an abnormality is made by the display unit 140, it may not be conveyed to a user at the site. A fifth modified example in which a patrol lamp is provided to notify the site of an abnormality will be described.

[0072] The determination system 1000 according to the fifth modification includes a patrol lamp instead of the projection device 300, and in step S209 of the determination process (FIG. 7), the control unit 110 notifies of an abnormality by rotating and emitting light from the patrol lamp.

[0073] In the fifth modification, the projection device 300 is not necessary, and the calibration process (step S104) is not necessary in the initial setting process (FIG. 3). This makes it possible to reduce the cost of the determination system and the effort required for installation.

[0074] (Other variations) Note that some of the above-mentioned embodiments and modifications may be combined in any way. For example, by combining Modification 1 and Modification 2, even if the objects are placed in the wrong order or facing the wrong direction, the control unit 110 can correctly determine whether or not an object is present, and if an object that should be there is missing, can notify the user that it is missing.

[0075] Similarly, by combining Modifications 1, 2, and 3, even if the objects are placed in the wrong order, facing the wrong direction, or have changed shape, the control unit 110 can correctly determine the presence or absence of the object, and if an object that should be there is missing, the user can be notified of that fact. Furthermore, by combining Modification 4, even if the color of the object changes over time, the presence or absence of the object can be correctly determined, and if an object that should be there is missing, the user can be notified of that fact.

[0076] In addition, the determination system 1000 is not limited to a system consisting of the determination device 100, the photographing device 200, the projection device 300, and the rotation device 400. For example, if all of the objects to be determined for their presence or absence fit into one photographed image, the rotation device 400 is not necessary. In addition, if the notification of an abnormality is made by displaying on the display unit 140 or by e-mail, the projection device 300 is not necessary. In addition, if the photographing device 200 has a control unit and a storage unit equivalent to the above-mentioned control unit 110 and storage unit 120, and the control unit of the photographing device 200 can perform the above-mentioned initial setting process and determination process, the determination device 100 is not necessary. In addition, the projection device 300 may have functions equivalent to the above-mentioned control unit 110 and storage unit 120, and if the projection device 300 can perform the above-mentioned initial setting process and determination process, the determination device 100 is not necessary.

[0077] Furthermore, the determination device 100 is not limited to a normal PC, and any computer can be used as long as it is a computer including the control unit 110, the storage unit 120, and the operation input unit 130. The determination device 100 may be, for example, a one-board microcomputer.

[0078] In the above embodiment, the programs for the initial setting process, the determination process, and the like executed by the control unit 110 have been described as being stored in advance in the storage unit 120. However, the programs may be stored in a non-transitory computer-readable recording medium such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), a DVD (Digital Versatile Disc), an MO (Magneto-Optical disc), a memory card, a USB memory, or the like, and distributed, and the programs may be read and installed in a computer to configure a computer capable of executing each of the above-mentioned processes.

[0079] Furthermore, the program may be superimposed on a carrier wave and applied via a communication medium such as the Internet. For example, the program may be distributed by posting it on a bulletin board system (BBS) on a communication network. The program may then be started and executed under the control of an OS (Operating System) in the same manner as other application programs, thereby enabling the above-mentioned processes to be performed.

[0080] In addition, the control unit 110 may be configured by any processor alone, such as a single processor, a multiprocessor, or a multi-core processor, or may be configured by combining any of these processors with a processing circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array).

[0081] Although the above describes a preferred embodiment of the present invention, the present invention is not limited to the specific embodiment, and the present invention includes the invention described in the claims and its equivalents. [Explanation of symbols]

[0082] 100...determination device, 110...control unit, 120...storage unit, 121...data for determination, 130...operation input unit, 140...display unit, 200...imaging device, 210, 220...determination image, 211, 212, 213...knife, 216, 217, 218...dotted line, 219, 229...object coordinates, 221, 222...driver, 300...projection device, 400...rotation device, 1000...determination system

Claims

1. A captured image is obtained by capturing an image of a predetermined area; determining whether a predetermined object is present in the captured image; When it is determined that the predetermined object does not exist, notifying that the object does not exist. A control unit is provided. Judgment device.

2. The control unit is projecting a predetermined image by a projection unit at a location where the object was present as information notifying that the object is not present; The determination device according to claim 1 .

3. The control unit is The photographed image is acquired as a judgment image at the time of initial setting, Thereafter, the photographed image is periodically acquired as a confirmation image, comparing the determination image with the confirmation image to determine whether the predetermined object is present; The determination device according to claim 2 .

4. The control unit is When the determination image is compared with the confirmation image, it is determined that the object is present even if the orientation of the object is different. The determination device according to claim 3 .

5. The control unit is When the determination image is compared with the confirmation image, it is determined that the object is present even if the location of the object is different. The determination device according to claim 3 .

6. The control unit is When the determination image and the confirmation image are compared, if an invariant portion, which is a portion of the object whose shape does not change, matches, it is determined that the object is present. The determination device according to claim 3 .

7. The control unit is When the determination image is compared with the confirmation image, if the only difference in color of the object is due to aging, the object is determined to be present. The determination device according to claim 3 .

8. The control unit: A captured image is obtained by capturing an image of a predetermined area; determining whether a predetermined object is present in the captured image; When it is determined that the predetermined object does not exist, notifying that the object does not exist. Judgment method.

9. In the control section, A captured image is obtained by capturing an image of a predetermined area; determining whether a predetermined object is present in the captured image; When it is determined that the predetermined object does not exist, notifying that the object does not exist. A program that executes a process.

Images