Folding method evaluation device, folding method evaluation method, and folding method evaluation program

Abstract

To provide a technique to support learning how to fold clothes without requiring a face-to-face instruction by a supervisor.SOLUTION: A folding evaluation apparatus includes detection means for detecting an action of folding clothes by a user based on sensor information, determination means for determining rightness of the detected action of folding the clothes based on a determination reference according to the clothes, and output means for outputting a determination result to an output apparatus.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] This disclosure relates to technologies for guiding how to fold clothes.

Background Art

[0002] Patent Document 1 describes a work support device that outputs, to an operator, a work procedure corresponding to the work content input by the operator so that the operator can confirm it using characters or video.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the invention of Patent Document 1, a worker to be trained does not need to be taught by a worker who is familiar with the work about the content of the work.

[0005] However, when an unfamiliar worker learns about how to fold clothes through characters or videos, there are angles that cannot be seen in the operation, which is difficult to understand. Also, it is difficult to know whether one's own folding method is correct.

[0006] An example of the object of this disclosure is to provide a technology for assisting in acquiring how to fold clothes.

Means for Solving the Problems

[0007] The folding method evaluation device in one aspect of this disclosure includes: a detection means for detecting an operation of folding clothes by a user based on sensor information; a determination means for determining the correctness of the detected operation of folding clothes based on a determination criterion corresponding to the clothes; and an output means for outputting the determination result to an output device.

[0008] A folding method evaluation method in one aspect of the present disclosure involves a computer detecting a user's action of folding clothes based on sensor information, determining the correctness of the detected clothing folding action based on judgment criteria corresponding to the clothing, and outputting the judgment result to an output device.

[0009] A folding method evaluation program in one aspect of this disclosure causes a computer to perform the following processes: detecting a user's action of folding clothes based on sensor information; determining the correctness of the detected clothing folding action based on judgment criteria corresponding to the clothing; and outputting the judgment result to an output device. [Effects of the Invention]

[0010] One example of the effect of this disclosure is that it can help people learn how to fold clothes. [Brief explanation of the drawing]

[0011] [Figure 1] This is a block diagram showing the configuration of the folding method evaluation device in the first embodiment. [Figure 2] This is an example of the output screen for the judgment result in the first embodiment. [Figure 3] This is an example of the output screen for the judgment result in the first embodiment. [Figure 4] This is an example of a database for outputting the judgment results in the first embodiment. [Figure 5] This is an example of the output screen for the judgment result in the first embodiment. [Figure 6] This is a flowchart illustrating the operation of the folding method evaluation device in the first embodiment. [Figure 7] This is a block diagram showing the configuration of the folding method evaluation system in the first embodiment. [Figure 8] This is a block diagram showing the configuration of the folding method evaluation device in the second embodiment. [Figure 9] This is an example of how the virtual space is displayed in the second embodiment. [Figure 10] This is an example of how the virtual space is displayed in the second embodiment. [Figure 11] This is an example of the display of the virtual space in the second embodiment. [Figure 12] This is an example of the display of the virtual space in the second embodiment. [Figure 13] This is a flowchart showing the operation of the folding method evaluation device in the second embodiment. [Figure 14] This is a block diagram showing the configuration of the folding method evaluation device in the third embodiment. [Figure 15] This is an example of the output screen of the determination result in the third embodiment. [Figure 16] This is an example of the display of the virtual space in the third embodiment. [Figure 17] This is an example of the display of the virtual space in the third embodiment. [Figure 18] This is an example of the display of the virtual space in the third embodiment. [Figure 19] This is a block diagram showing the configuration of the folding method evaluation device in the fourth embodiment. [Figure 20] This is an example of the display of the virtual space in the fourth embodiment. [Figure 21] This is an example of the display of the virtual space in the fourth embodiment. [Figure 22] This is an example of the display of the virtual space in the fourth embodiment. [Figure 23] This is a diagram showing the hardware configuration in which the folding method evaluation device in the present disclosure is realized by a computer device and its peripheral devices. [Embodiments for Carrying Out the Invention]

[0012] Embodiments of the present disclosure will be described in detail with reference to the drawings. In the present disclosure, each folding method evaluation device is used, for example, in folding method training. Alternatively, the folding method evaluation device may be used in evaluation situations such as tests and contests other than training.

[0013] [First Embodiment] Figure 1 is a block diagram showing the configuration of the folding method evaluation device 100 in the first embodiment. Referring to Figure 1, the folding method evaluation device 100 comprises a detection unit 101, a determination unit 102, and an output unit 103. The detection unit 101 detects the user's action of folding clothes based on sensor information. The determination unit 102 determines the correctness of the detected action based on determination criteria corresponding to the clothes. The output unit 103 outputs the determination result to an output device. The detection unit 101, the determination unit 102, and the output unit 103 may be implemented by a combination of hardware such as a computer or mobile communication terminal including a processor, and firmware or software.

[0014] Next, the configuration of the folding method evaluation device 100 in the first embodiment will be described in detail.

[0015] In Figure 1, the detection unit 101 is an example of a detection means that detects the action of a user folding clothes based on sensor information. The user is a person who uses the folding method evaluation device 100. The sensor information is, for example, an image of the user folding clothes. In this case, the detection unit 101 acquires an image of the user folding clothes from a camera that is communicatively connected to the folding method evaluation device 100. The detection unit 101 then detects the user folding clothes by analyzing the acquired image. The detection unit 101 detects the user's actions by, for example, detecting the position and movement of the wrist joints and finger joints of a person included in the image. Alternatively, the detection unit 101 detects the user's actions by detecting the skeletal state of the hands and fingers of a person included in the image. The detection unit 101 may also detect the movements of the user's hands, arms, and body, including their gaze.

[0016] The image is a real-time image captured by a camera that records the action of folding clothes. Alternatively, the image may be a pre-recorded image of the clothes-folding action captured by the camera. In this case, the image is stored in memory (not shown).

[0017] In this embodiment, the user's action of folding clothes may be completed in a single step or in multiple steps. Specifically, the detection unit 101 detects the action of folding clothes step by step based on the image, which is sensor information. That is, as described above, the detection unit 101 detects the user's action of folding clothes step by step by detecting the position and movement of the user's wrist joints and finger joints included in the image. Alternatively, the detection unit 101 detects the user's action of folding clothes step by step by detecting the state of the hand and finger skeletons.

[0018] Next, an example of detection will be described. In this example, the detection unit 101 detects the action of folding clothes using a trained model. For example, the detection unit 101 inputs an image of the action of folding clothes into a trained model that has been trained to identify the action of folding clothes. Next, the detection unit 101 uses the trained model to identify the position and movement of each joint, or the state of the bones of the hand and fingers, as actions from the input image. Furthermore, the detection unit 101 detects action features representing the actions identified by the trained model at each step. The detection unit 101 then outputs the action features for each step of folding clothes to the determination unit 102 as the detection result of the action of folding clothes. Action features are features extracted from the action. The detection of the action of folding clothes by the detection unit 101 is not limited to this and may be performed by other methods.

[0019] The camera used to film the user folding clothes is, for example, a surveillance camera (not shown) installed in a store, or a camera (not shown) installed in the folding method evaluation device 100. Furthermore, multiple cameras may be used to film the user folding clothes. By using images captured by multiple cameras, the detection unit 101 can eliminate camera blind spots. For example, the sensor information may be the images from each camera. In this case, the detection unit 101 detects the user's movements from each image captured at the same time. For example, the sensor information may be a composite image of the images from each camera. In this case, the detection unit 101 detects the user's movements from the composite image. Also, the sensor information may be images of the clothes being folded taken from different directions by multiple cameras. This allows the detection unit 101 to more accurately detect movements in each direction.

[0020] The determination unit 102 is an example of a determination means that determines the correctness of the detected clothing folding action based on determination criteria corresponding to the clothing. For example, the determination unit 102 may determine whether or not the clothing folding action is correct. Alternatively, for example, the determination unit 102 may determine whether or not multiple items are correct and determine the correctness based on the total number of correct or incorrect answers.

[0021] For example, the determination unit 102 obtains judgment criteria corresponding to the clothing from a storage unit (not shown). The storage unit (not shown) stores judgment criteria for each type of clothing, as will be described later. The storage unit is provided to communicate with the inside or outside of the folding evaluation device 100. The detection or estimation of the type of clothing for obtaining the judgment criteria corresponding to the clothing will be described later. After the detection unit 101 has detected the user's actions in folding the clothing and obtained the judgment criteria corresponding to the clothing, the determination unit 102 compares the two. Furthermore, the determination unit 102 determines the correctness of the user's actions in folding the clothing according to the comparison result. For example, the comparison result may be the similarity between a feature representing the user's actions in folding the clothing and a feature representing the judgment criteria corresponding to the clothing. Alternatively, the comparison result may be the difference between information indicating the user's actions in folding the clothing and information representing the judgment criteria corresponding to the clothing. If the similarity or difference is greater than a threshold, the determination unit 102 determines that the user's actions in folding the clothing are incorrect, that is, the user's actions in folding the clothing are wrong. The actions performed by the determination unit 102 to determine the correctness of the user's actions in folding clothes are not limited to these.

[0022] The criteria for judging clothing are, for example, information that associates the type of clothing with the criteria for judging. The type of clothing may be a group classified by different folding methods. The type of clothing may be divided by, for example, sleeve length, presence or absence of a hood, presence or absence of a collar, and size. The type of clothing whose folding method is evaluated by the user is determined, for example, by user or user administrator input. When a user evaluates the folding method of clothing, they may first input the type of clothing through user input. The input is performed, for example, by the input device of the computer that constitutes the folding method evaluation device 100. The criteria for judging are reference information for comparison with the aforementioned operation features in the judgment unit 102. Also, if the operation is represented by an operation feature, the reference information is the reference operation feature. In the following description, the reference operation feature will be called the reference operation feature.

[0023] For example, the reference motion features are generated using images of the correct movements of a model worker folding clothes, as follows: The reference motion features are generated by the folding method evaluation device 100 or other information processing device. The generated reference motion features are stored in the memory unit.

[0024] To generate baseline motion features, the position and movement of the wrist and finger joints of the model worker in the image are detected step by step. Alternatively, the skeletal state of the hand and fingers of the model worker in the image is detected step by step.

[0025] Let's describe an example of generating baseline motion features. For example, images of a model worker folding clothes are input into the aforementioned trained model that has learned the motion of folding clothes. The trained model, with the model worker's images as input, identifies the position and movement of each joint of the model worker's wrist and fingers, or the state of the skeletal structure of the hand and fingers, as motions. The identified motion features representing the motion are detected for each step. Then, the motion features for each step of folding clothes are output, or generated, as baseline motion features. Note that the generation of baseline motion features is not limited to this example.

[0026] A baseline operating feature is generated for each type of clothing. This baseline operating feature is then stored in the memory unit, associated with the type of clothing.

[0027] To obtain judgment criteria based on the type of clothing, the judgment unit 102 may recognize the type of clothing from the image captured by the camera. For example, the judgment unit 102 recognizes the clothing included in the image captured by the camera using image recognition technology such as a trained model. Then, the judgment unit 102 identifies the type of clothing based on the information associated with the recognized clothing. Alternatively, for example, the judgment unit 102 may recognize the shape of the clothing included in the image captured by the camera. Then, the judgment unit 102 may detect or estimate the type of clothing based on the recognized shape. The color of the clothing may also be used for detecting or determining the type of clothing. The judgment unit 102 then obtains judgment criteria corresponding to the type of clothing from a database. Subsequently, the judgment unit 102 determines the correctness of the user's action of folding the clothing based on the judgment criteria corresponding to the type of clothing.

[0028] The determination unit 102 may determine the correctness of each step of the clothes-folding operation based on the determination criteria set for each step. Alternatively, the determination unit 102 may determine the correctness of each step after the completion of the series of clothes-folding operations. Or, the determination unit 102 may determine the correctness of each step after each step of the operations has been completed. Note that the series of clothes-folding operations refers to the operation of folding one or more pieces of clothing.

[0029] The judgment criteria handled by the judgment unit 102 may be a list of evaluation items for evaluating correct operation. An evaluation item is an item for determining correctness. An evaluation item is set for each operation. For example, evaluation items are the correct place for the user to hold the garment, and the places where parts of the garment, such as the hem, sleeves, shoulders, and sides, should be in an operation with multiple steps. These locations can be expressed by their positional relationship to part or all of the garment. A list containing information on multiple evaluation items is stored as a judgment criterion in the folding evaluation device 100 or an externally connectable storage unit.

[0030] Furthermore, as a judgment criterion, information indicating the correct line for folding clothes, the time limit for the action, or the correct order of actions for each of multiple steps may be stored in a list of evaluation items for correct action. In this case, the judgment unit 102 determines whether the image matches the information indicating the evaluation item associated with each action. For example, if the correct place for the user to hold the clothes is set as an evaluation item, the judgment unit 102 determines whether the position of the user's hands and clothes in the image is in the correct place. Also, for example, if the time limit for the action is set as an evaluation item, the judgment unit 102 may determine whether the time spent on the action is within the time limit. The judgment unit 102 calculates the time spent on the action, for example, from the start time and end time of the action detected from the image. Note that the judgment criterion may set a range in which the action is judged to be correct.

[0031] Furthermore, the determination unit 102 may determine the state of the folded clothes in addition to the user's actions of folding the clothes. In this case, the detection unit 101 further detects the state of the clothes folded by the user from the image captured of the user's actions of folding the clothes. The determination unit 102 may then determine the correctness of the state of the clothes after each action in each of the multiple steps has been performed. The criteria for determining the state of the clothes folded by the user may be, for example, the position of parts of the clothes folded by a model worker, such as the hem, sleeves, shoulders, and sides, relative to other parts, or a reference feature representing the width or length in the back direction of the folded clothes.

[0032] The determination unit 102 may correct its determination of the user's action of folding clothes based on its determination of the state of the clothes folded by the user. For example, if the determination unit 102 determines that the action of folding clothes is correct but the state of the clothes is incorrect, it may correct its determination of the user's action of folding clothes by determining that the action is incorrect.

[0033] The output unit 103 is an example of an output means for outputting the judgment result to an output device. The output unit 103 outputs the judgment result determined by the judgment unit 102 to an output device used by the user or the user's administrator. For example, the output device is a wearable display worn by the user, or a display device that the user can view. In this case, the output unit 103 causes the output device to display the judgment result. Alternatively, the output device may be an audio output device. In this case, the output unit 103 may output the judgment result to the audio output device. Alternatively, the output unit 103 may output the judgment result not only to an output device, but also to a database that records the user's evaluation of the folding method.

[0034] The output unit 103 outputs information indicating that the user's actions in folding clothes are correct as a judgment result to the output device if there are no errors in the user's actions in folding clothes. Figure 2 shows an example of the output screen when there are no errors in the user's actions in folding clothes. In the example output screen in Figure 2, "Judgment Result: CLEAR" is displayed. The output screen may also display the type of clothing being folded (in Figure 2, "Clothing being folded: Short-sleeved T-shirt") and user information (in Figure 2, "Employee Number XXX Name YYY"). Furthermore, the output screen may also display operation buttons such as "Fold Next Clothes" and "Return to Home" as shown in Figure 2. The output of the judgment result is not limited to this example.

[0035] Furthermore, if the user makes an error in folding clothes, the output unit 103 outputs information indicating that the operation is incorrect as a judgment result to the output device. The judgment result that the output unit 103 outputs to the output device to the user is at least one of the following: correctness of the operation, correctness of each operation in each of the multiple steps, an incorrect operation, or the correct operation corresponding to the incorrect operation. For example, the output unit 103 may output a video of the correct operation corresponding to the incorrect operation to the output device. The output unit 103 may output the judgment result to the output device after the series of operations for folding clothes is completed. Alternatively, the output unit 103 may output the judgment result to the output device for each of the multiple steps.

[0036] Figure 3 shows an example of the output screen when the user makes an error in folding clothes. The example output screen in Figure 3 displays "Judgment Result: MISS". The output screen may also display the action that was incorrect, such as "There was a mistake in step 3" in Figure 3. The output screen may also display how the user made a mistake in the incorrect action, such as "In step 3, you should hold the right hem, not the right sleeve". Furthermore, the output screen may display operation buttons to check the correct procedure, such as "Check the procedure in a video," as shown in Figure 3, and operation buttons to restart the evaluation, such as "Try again." In addition, even when the user makes an error in folding clothes, the output screen may display the type of clothing being folded, user information, and an operation button to return to the initial screen, similar to Figure 2. Note that the output of the judgment result is not limited to this example.

[0037] Furthermore, for example, the output unit 103 may output the judgment results of the user's clothes-folding actions to a database accessible to the user's administrator. The database accessible to the user's administrator is, for example, a database that stores employee information. Also, for example, the user's administrator is someone in a position to manage employees. This allows the user's administrator to know the progress of the user's clothes-folding training if the user is undergoing training. The judgment results output to the database accessible to the user's administrator are, for example, the judgment results for each action, the judgment results for a series of actions, the final judgment result for the day, the latest judgment result, and the progress relative to the overall training, at least one of these. In addition, the start date of the training, the number of days elapsed since the start date of the training, and the total training time may be associated with and stored in the database in relation to the judgment results. In this case, the judgment results may also be called the training history.

[0038] Figure 4 shows an example of a database where the judgment results for an employee user are stored. The example database in Figure 4 stores employee identification information such as employee number and employee name. The example database in Figure 4 also stores the training progress for each employee as a judgment result. In this case, the output unit 103 calculates the progress rate of the user relative to the overall training. The output unit 103 then outputs an instruction to the database to rewrite the progress rate associated with the user's identification information with the calculated progress rate. The progress rate may be, for example, a value that shows as a percentage how many times the type of clothing that was judged to be correct when folding clothes is determined to be correct is included out of the total types of clothing in the training. Note that the database that the user's administrator can check is not limited to this example. Also, the training progress is not limited to the progress rate described above.

[0039] Alternatively, the output unit 103 may output the judgment result of the user's clothes-folding action to a database accessible to the user's administrator, without outputting the judgment result to the user. For example, if a user uses the clothes-folding evaluation device 100 for a training test on how to fold clothes, it is sufficient for only the user's administrator to be notified of the judgment result.

[0040] Furthermore, the output unit 103 may output to the output device that it was not possible to detect the action of folding clothes, or that it was possible to detect the action of folding clothes but not to determine that it was a folding action. In this case, the output unit 103 may also output to the output device information instructing the user to improve the shooting environment so that the detection or determination of the action of folding clothes can be performed normally. For example, the output unit 103 may output to the output device the user's position relative to the camera or the camera's installation position.

[0041] Figure 5 shows an example of the output screen when the folding method evaluation device 100 fails to detect that the user is folding clothes. In Figure 5, the output screen indicates that the folding clothes could not be detected and that the folding action could not be determined. The output screen may also display instructions for improving the shooting environment, such as "Please brighten the shooting environment" as shown in Figure 5. Furthermore, even if the folding action of clothes cannot be detected or determined, the output screen may, as in Figure 2, display the type of clothes being folded, user information, an operation button to evaluate the folding method of the same clothes again, and an operation button to return to the initial screen.

[0042] The operation of the folding method evaluation device 100, configured as described above, will be explained with reference to the flowchart in Figure 6.

[0043] Figure 6 is a flowchart illustrating the operation of the folding method evaluation device 100 in the first embodiment. Note that the processing shown in this flowchart may be executed based on program control by a processor.

[0044] As shown in Figure 6, first, the detection unit 101 detects the user's action of folding clothes (step S101).

[0045] Next, the determination unit 102 determines whether the user's action of folding clothes, detected in step S101, is correct or not, based on determination criteria corresponding to the clothes (step S102).

[0046] Next, the output unit 103 outputs the determination result, which is the result of the determination in step S102, to the output device (step S103).

[0047] With this, the folding method evaluation device 100 completes its series of operations. Steps S101 to S103 may also be performed for each of the operations divided into multiple steps. Furthermore, each step and each combination of steps may be repeated.

[0048] In the folding method evaluation device of the embodiment described above, the detection unit detects the user's action of folding clothes based on sensor information. The determination unit determines the correctness of the detected action based on determination criteria corresponding to the clothes. The output unit outputs the determination result to the output device.

[0049] As a result, the folding method evaluation device in this embodiment can assist in learning how to fold clothes.

[0050] [Example of application of the first embodiment] One example of the application of the first embodiment is a folding method evaluation system 10 used in a store. In the store, a surveillance camera may be used to capture images for detecting motion. Note that the surveillance camera is just one example of a camera that captures images for detecting motion, and cameras other than surveillance cameras may be used.

[0051] Figure 7 is a block diagram showing the configuration of the folding method evaluation system 10 when the folding method evaluation device 100 is used in a store. The folding method evaluation system 10 comprises the folding method evaluation device 100, a surveillance camera 110 installed in the store, and a display device 120.

[0052] The folding method evaluation device 100 acquires images of the user folding clothes, captured by the surveillance camera 110, as sensor information.

[0053] The trigger for the surveillance camera 110 to transmit images of the clothing folding action to the clothing folding evaluation device 100 is, for example, when the surveillance camera 110 recognizes a specific action performed by the user based on the image. When the surveillance camera 110 recognizes a specific action by the user, it transmits images of the clothing folding action to the clothing folding evaluation device 100.

[0054] Alternatively, the trigger for the surveillance camera 110 to transmit images of the clothing folding action to the folding method evaluation device 100 may be communication with a terminal owned by the user. When wireless communication becomes possible with the user's dedicated terminal for evaluating the folding method, the surveillance camera 110 transmits images of the clothing folding action to the folding method evaluation device 100. Alternatively, if the user presses a button on their terminal to initiate the folding method evaluation, the surveillance camera 110, upon receiving the start instruction, may transmit images of the clothing folding action to the folding method evaluation device 100. The start instruction is notified to the surveillance camera 110 either directly from the user's terminal or through the folding method evaluation device 100.

[0055] The display device 120 is an example of an output device that outputs the judgment result. The display device 120 is, for example, a dedicated terminal for evaluating folding methods owned by the user, a terminal owned by the user, or a glasses-type display device capable of displaying the judgment result. If the display device 120 is a glasses-type display device capable of displaying the judgment result, the user wears the glasses-type display device capable of displaying the judgment result to have their folding method evaluated. The glasses-type display device capable of displaying the judgment result is a device that can display information such as the judgment result on glasses that allow the user to see their surroundings. In this way, by having the user wear a wearable terminal capable of outputting the judgment result, the user does not need to carry a terminal to check the judgment result. In particular, with a wearable display device 120 such as glasses, the user does not need to carry a terminal to visually check the judgment result.

[0056] Furthermore, as another application example of the first embodiment, the folding method evaluation device 100 may be implemented using smart glasses with a built-in camera.

[0057] [Second Embodiment] Next, a second embodiment of this disclosure will be described in detail with reference to the drawings. The second embodiment differs from the first embodiment in that the user performs the action of folding virtual clothes displayed in a virtual space. To the extent that the description of this embodiment does not become unclear, explanations that overlap with the above description will be omitted.

[0058] Figure 8 is a block diagram showing the configuration of a folding method evaluation device 200 according to a second embodiment of the present disclosure. The folding method evaluation device 200 of the second embodiment includes a display control unit 204 in addition to the configuration of the first embodiment. Furthermore, the folding method evaluation device 200 includes a detection unit 201, a determination unit 202, and an output unit 203, respectively, replacing the detection unit 101, determination unit 102, and output unit 103 of the first embodiment.

[0059] First, the display control unit 204 is an example of a display control means that displays virtual clothes folded by the user in a virtual space. The display control unit 204 displays the virtual space on a display device, such as a head-mounted display, a dome-shaped display, or an aerial display.

[0060] The display control unit 204 may, for example, display a type of virtual clothing selected by the user on the display device screen. Alternatively, the display control unit 204 may select the type of virtual clothing to display. For example, the display control unit 204 may select the type of virtual clothing to display based on the user's training history. The display control unit 204 may select from the training history a type of virtual clothing in which the user previously made an incorrect action as a review. Alternatively, the display control unit 204 may select a type of virtual clothing in which the user has not yet trained.

[0061] The display control unit 204 changes the state of the virtual clothing displayed on the display device based on the action of folding the virtual clothing detected by the detection unit 201. For example, the display control unit 204 may search a database for the next state of the virtual clothing to be displayed based on the state of the virtual clothing currently displayed on the display device and the action detected by the detection unit 201. Alternatively, for example, the display control unit 204 may calculate the next state of the virtual clothing to be displayed from the currently displayed state of the virtual clothing and the detected action. In this case, for example, the ease of movement, stretchability, twistability, and lightness of each position of the virtual clothing are set. The display control unit 204 calculates the position of each point of the output virtual clothing based on the input action.

[0062] Furthermore, if the determination unit 202 determines that the user's action of folding the virtual clothes is incorrect, the display control unit 204 may revert the display showing the state of the virtual clothes to the state it was in before the incorrect action occurred.

[0063] The display control unit 204 may display a virtual hand of the user folding clothes. The virtual hand is displayed in a virtual space and represents the movement of the user's hand. In this case, the display control unit 204 changes the state of the virtual hand to be displayed based on the action detected by the detection unit 201. That is, the display control unit 204 changes the state of the virtual hand to be displayed so that the virtual hand moves in conjunction with the movement of the user's hand. Alternatively, the display control unit 204 may display the actual hand of the user folding clothes, which has been extracted from a captured image. In this case, the displayed user's hand may be, for example, an image of only the hand portion captured by a camera provided in the display device, which is then displayed on the screen of the display device.

[0064] An example of a screen that the display control unit 204 displays on the display device will be described below. In the following description, at least one of the right hand and the left hand is either the actual hand or a virtual hand as described above.

[0065] First, Figure 9 is an example of a screen that the display control unit 204 displays on the display device in the second embodiment when the user starts training to fold virtual clothes. In the center of Figure 9, a virtual short-sleeved T-shirt is displayed. Also, the initial screen example in Figure 9 shows the user's right and left hands. For example, if the user moves their hands, the "Start" operation button is selected in Figure 9. The detection unit 201 may start detecting the action based on this operation. In addition, as shown in the display of "short-sleeved T-shirt" in Figure 9, the type of clothing displayed in the virtual space and for which the user's folding method is evaluated may be displayed.

[0066] Next, Figure 10 shows an example of a screen that the display control unit 204 displays on the display device when the user is performing the action of folding virtual clothes in the second embodiment. The example screen in Figure 10 is an example of the display after the initial screen in Figure 9, when the user's left and right hands grasp both shoulders of a short-sleeved T-shirt and then lift it up. As mentioned above, the display of the user's left and right hands and the virtual clothes is changed based on the action of folding clothes by the user, which is detected by the detection unit 201. Also, while folding clothes as in Figure 10, an operation button for the user to temporarily suspend the training may be displayed, as shown in the upper right of Figure 10.

[0067] Next, Figure 11 is an example of a screen that the display control unit 204 displays on the display device when the user makes an error in folding the virtual clothes in the second embodiment. In addition to the user's left and right hands and the virtual clothes, Figure 11 shows that the user made an error in folding the virtual clothes. Furthermore, if the user makes an error in folding the virtual clothes, information indicating how the action was done incorrectly, such as "Incorrect procedure," and operation buttons to confirm the correct procedure may be displayed, as shown in Figure 11.

[0068] Furthermore, Figure 12 is an example of a screen that the display control unit 204 displays on the display device when it is determined that the user's action of folding the virtual clothes is correct in the second embodiment. In Figure 12, the display control unit 204 displays the user's left and right hands and the virtual clothes, as well as information indicating that it has been determined that the user's action of folding the virtual clothes is correct. Also, when it is determined that the user's action of folding the virtual clothes is correct, operation buttons may be displayed, as in Figure 12, for example, to continue the folding training, to perform the same clothing folding training again, or to return to the initial screen.

[0069] The detection unit 201 detects the user's actions of folding virtual clothes displayed in a virtual space based on sensor information. Sensor information is, for example, information acquired by motion sensors attached to the user's body. For example, a motion sensor consists of at least one of an acceleration sensor, a gyroscope sensor, or a geomagnetic sensor. An acceleration sensor is a sensor that measures acceleration that occurs when there is a change in movement. A gyroscope sensor is a sensor that measures angular velocity, which is the speed of rotation. A geomagnetic sensor is a sensor that measures the direction of the geomagnetic vector. For example, the user wears motion sensors on parts of their body that move when folding clothes, such as their arms, hands, or fingers. For example, by having the user wear gloves equipped with motion sensors and perform the action of folding clothes, the detection unit 201 detects the user's actions of folding clothes based on the motion sensor information.

[0070] The detection unit 201 may combine the detection of the user folding clothes using an image, as described in the first embodiment, with the detection of the user folding clothes using motion sensor information. Furthermore, the detection of the user folding clothes using motion sensor information may be applied to the detection unit 101 of the first embodiment, which detects the folding of actual clothes rather than virtual clothes.

[0071] The determination unit 202 determines the correctness of the user's action of folding the virtual clothing based on the determination criteria corresponding to the virtual clothing detected by the detection unit 201. The determination criteria corresponding to the virtual clothing may be associated with the information of the virtual clothing displayed by the display control unit 204. Alternatively, the determination unit 202 may obtain the determination criteria corresponding to the information of the type of virtual clothing displayed by the display control unit 204 from a storage unit in which the type of clothing and the determination criteria are associated and stored.

[0072] The output unit 203 outputs the judgment result to an output device. For example, the output unit 203 displays the judgment result in a virtual space where virtual clothes are folded. Alternatively, the output unit 203 may output the judgment result to a database that can be viewed by the user's administrator, similar to the first embodiment.

[0073] The operation of the folding method evaluation device 200, configured as described above, will be explained with reference to the flowchart in Figure 13.

[0074] Figure 13 is a flowchart illustrating the operation of the folding method evaluation device 200 in the second embodiment. Note that the processing shown in this flowchart may be executed based on program control by a processor.

[0075] As shown in Figure 13, first the display control unit 204 displays the virtual clothes that the user is folding (step S201).

[0076] Next, the detection unit 201 detects the user's action of folding virtual clothes (step S202).

[0077] Next, the display control unit 204 changes the state of the virtual clothing to be displayed based on the user's actions detected in step S202 (step S203).

[0078] Next, the determination unit 202 determines whether the user's action of folding the virtual clothes is correct or not, based on the determination criteria corresponding to the virtual clothes (step S204).

[0079] Next, the output unit 203 outputs the determination result from step S204 to the output device (step S205).

[0080] With this, the folding method evaluation device 200 completes its series of operations. Steps S201 to S205 may also be performed for each of the operations divided into multiple procedures. Furthermore, each step and each combination of steps may be repeated.

[0081] In the folding method evaluation device of the embodiment described above, the display control unit displays a virtual garment. The detection unit detects the user's actions of folding the garment based on sensor information. The determination unit determines the correctness of the detected actions based on determination criteria corresponding to the garment. The output unit outputs the determination result to the output device.

[0082] As a result, the folding method evaluation device in this embodiment can assist in learning how to fold clothes. In particular, the folding method evaluation device in this embodiment can evaluate how clothes are folded without having to prepare actual clothes in a real space.

[0083] [Modified version of the second embodiment] In the second embodiment, the user folds virtual clothing in a virtual space. However, in the second embodiment, real-world clothing may be used. Real-world clothing is clothing that the user can pick up in the real space, not in a virtual space.

[0084] In this case, the user uses real-world clothing to perform training with the folding evaluation device of the second embodiment. Alternatively, for example, based on the state of the real-world clothing captured by a camera, the display control unit 204 may display a virtual garment in a state linked to the folding operation of the real-world clothing. This allows the display control unit 204 to more accurately change the state of the virtual garment to be displayed. Furthermore, the real-world clothing used in this case may be equipped with the motion sensor described in the second embodiment. The display control unit 204 may change the state of the virtual garment to be displayed based on the sensor information from the motion sensor equipped on the real-world clothing.

[0085] The clothing in real space may be other types of fabric. Furthermore, the type of clothing in real space and the type of virtual clothing displayed by the display control unit 204 may be different. By using real-world clothing or fabric, the user can evaluate various types of virtual clothing folding methods without having to prepare various types of clothing themselves. This allows the user to naturally evaluate clothing folding methods while retaining the sensation of folding fabric.

[0086] [Third Embodiment] Next, a third embodiment of this disclosure will be described in detail with reference to the drawings. The third embodiment differs from the first and second embodiments in that the folding method evaluation device 300 detects and determines the user's action of bagging clothes in addition to the user's action of folding clothes. To the extent that the description of this embodiment does not become unclear, explanations that overlap with the above description will be omitted.

[0087] In this embodiment, the bagging action refers to the action of putting clothes into a bag. For example, the bagging action refers to the action of putting clothes, which are purchased items, into a shopping bag. Alternatively, the bagging action may refer to the action of putting clothes, which are products, into a shipping bag.

[0088] Figure 14 is a block diagram showing the configuration of a folding method evaluation device 300 according to a third embodiment of the present disclosure. The folding method evaluation device 300 of the third embodiment includes a detection unit 301, a determination unit 302, and an output unit 303. The folding method evaluation device 300 may also include a display control unit 304 (not shown).

[0089] In addition to the functions of detection unit 101 or detection unit 201, detection unit 301 also detects the user's action of bagging items. The detection of bagging actions may be performed in the same way as the detection of the action of folding clothes. Furthermore, detection unit 301 may also detect the state of the bag and the clothes, similar to the detection of the state of the clothes described above.

[0090] The determination unit 302, in addition to the functions of the determination unit 102 or determination unit 202, further determines the correctness of the user's bagging action. The determination of the bagging action may be performed in the same way as the determination of the clothing folding action. For example, the determination unit 302 determines the correctness of the bagging action detected by the detection unit 101 or detection unit 201 based on determination criteria corresponding to the type and number of garments. The determination criteria corresponding to the type and number of garments are the appropriate type and number of bags for the type and number of garments to be bagged. Furthermore, the appropriate type and number of bags are not limited to one or one combination. In the case where the user is practicing folding using real-world garments, as in the first embodiment, the type and number of garments may be detected or estimated from images of the user's actions.

[0091] Furthermore, the determination unit 302 may determine the correctness of the user's bag selection when determining the bagging operation. The correctness of the bag selection is whether the bag is appropriate for the clothes to be placed in the bag without tearing. For example, the bag selection is the selection of a shopping bag for the clothes. In this case, there may be one or more clothes.

[0092] The determination unit 302 may determine the correctness of the user's bag selection based, for example, on information about appropriate bags associated with the type of clothing. The information about appropriate bags includes information indicating the appropriate type and number of bags for the type of clothing. The type of bag is the size of the bag. Alternatively, the type of bag may include the material, color, pattern, and shape of the bag. The type of clothing associated with the information about appropriate bags may also be a combination of clothing. The combination of clothing includes the type of clothing and the number of items of each type. For example, if the clothing to be bagged is a combination of two short-sleeved T-shirts and one down jacket, the information about appropriate bags associated with it may be a large paper bag.

[0093] Alternatively, the determination unit 302 may determine the correctness of the user's bag selection based on information about the appropriate bag determined for the clothing to be bagged. The method for determining the appropriate bag for the clothing is not limited.

[0094] The appropriate bag for the clothes to be bagged may be determined by the determination unit 302 or an external information processing device. For example, the determination unit 302 or an external information processing device may identify the size of the clothes to be bagged. The determination unit 302 or an external information processing device may identify, for example, the type and number of clothes. Next, the determination unit 302 or an external information processing device may retrieve the size information stored in association with the identified type of clothes. Then, the determination unit 302 or an external information processing device may calculate the total size of all the clothes to be bagged.

[0095] Alternatively, the determination unit 302 or an external information processing device measures or estimates the size of the garment to be bagged. The measurement or estimation may be performed by a sensor or image recognition. Here, size refers to at least one of volume, length, or weight. The volume of the garment may be the volume after the garment has been folded. The determination unit 302 or the external information processing device then determines the type and number of bags according to the size of the identified garment.

[0096] The determination unit 302 or an external information processing device should determine the appropriate bag type if the size of the identified garment falls within the range of sizes that can be placed in the bags, which are stored in association with each bag type. At least an upper limit should be set for the size range. If the size of the identified garment exceeds the range of sizes that can be placed in the bags, the determination unit 302 or an external information processing device should determine several appropriate bag types and the number of garments for each bag type. The determination unit 302 or an external information processing device may also store the information on the garment to be bagged and the appropriate bags, as determined in this way, in a database.

[0097] The output unit 303 outputs the judgment results for the clothes folding operation and the bagging operation to the output device. The content and destination of the output are the same as in the first and second embodiments. Figure 15 is an example of an output screen that the output unit 303 has the output device output. In Figure 15, the output unit 303 outputs to the output device that the operation was incorrect as a result of the judgment. The output unit 303 may also output the judgment results for the clothes folding operation and the bagging operation to the output device, as shown in Figure 15. In addition, if the bag selection is judged and the user's bag selection is incorrect, the output unit 303 may output to the output device that the bag selection is incorrect. Furthermore, the output unit 303 may output to the output device that the bagging operation has been detected and judged, as shown in Figure 15.

[0098] Furthermore, if the folding method evaluation device 300 of the third embodiment is such that the user folds virtual clothing, as in the second embodiment, the folding method evaluation device 300 includes a display control unit 304 (not shown).

[0099] The display control unit 304 displays a virtual bag in addition to the virtual clothing on the display device. Figure 16 is an example of a screen displayed by the display control unit 304 on the display device. In Figure 16, the display control unit 304 displays virtual clothing, the user's virtual hand, and a virtual bag.

[0100] Similar to the second embodiment, the display control unit 304 may display a virtual bag of a type selected by the user. Alternatively, the display control unit 304 may select a type of virtual clothing to display. For example, the display control unit 304 may display bags of an appropriate type and number for the virtual clothing being displayed, i.e., the virtual clothing to be bagged.

[0101] Furthermore, the display control unit 304, similar to the display control unit 204 in the second embodiment, may change the state of the virtual bag to be displayed based on the user's actions toward the virtual bag detected by the detection unit 301. For example, if the user performs an action to open the bag, the state of the virtual bag will be displayed as open.

[0102] Furthermore, the display control unit 304 may display options for the user to select a virtual bag. This allows the folding evaluation device 300 to detect and determine the user's bag selection in the virtual space. Figures 17 and 18 are examples of display screens for bag options. In Figure 17, when the user selects "Bag Selection," the display control unit 304 displays options for bag types in Figure 18. In the example in Figure 18, the display control unit 304 displays the bag material and bag size as options. The examples of bag options are not limited to these. For example, the display control unit 304 may display virtual shelves and multiple types of virtual bags stocked on the virtual shelves as bag options, similar to a real store. The display control unit 304 may also display options for the user to select a virtual bag after the user has finished folding the displayed virtual clothes.

[0103] In the folding method evaluation device of the embodiment described above, the detection unit detects the user's actions of folding clothes and bagging them based on sensor information. The determination unit determines the correctness of the detected actions based on determination criteria corresponding to the clothes. The output unit outputs the determination result to the output device.

[0104] As a result, the folding method evaluation device in this embodiment can assist in learning how to fold clothes. In particular, the folding method evaluation device in this embodiment can assist in evaluating not only how to fold clothes but also how to pack them into bags.

[0105] [Fourth Embodiment] Next, a fourth embodiment of this disclosure will be described in detail with reference to the drawings. The fourth embodiment differs from the embodiments described above in that the folding method evaluation device 400 determines the user's actions in folding clothes according to the situation. To the extent that the description of this embodiment does not become unclear, explanations that overlap with the above descriptions will be omitted.

[0106] In this embodiment, the situation refers to the circumstances surrounding the user folding clothes.

[0107] Figure 19 is a block diagram showing the configuration of a folding method evaluation device 400 according to the fourth embodiment of the present disclosure. The folding method evaluation device 400 of the fourth embodiment includes a detection unit 401, a determination unit 402, and an output unit 403. The folding method evaluation device 400 may also include a display control unit 404 (not shown).

[0108] The determination unit 402 sets determination criteria according to the situation in which the clothes are folded. That is, the determination unit 402 determines the correctness of the action based on the determination criteria according to the situation in which the clothes are folded and the type of clothes. The determination criteria according to the situation may be set in the same way as in the first embodiment.

[0109] When a user folds clothes in real space, for example, the determination unit 402 identifies the situation based on the user's surrounding image. For example, the determination unit 402 recognizes the surrounding environment in which the user folds clothes from the surrounding image of the area where the clothes-folding work is performed (hereinafter referred to as the surrounding image). The determination unit 402 identifies the recognized surrounding environment as a situation. Then, the determination unit 402 sets judgment criteria according to the identified situation. Alternatively, when a user folds virtual clothes, the determination unit 402 sets judgment criteria according to the situation selected by the user or the display control unit 404.

[0110] The criteria for judgment can be set according to different situations in which the user folds clothes. For example, a situation could be one in which there is a table available for the user to use to fold clothes, or one in which there is no table. That is, the user's action of folding clothes can be either using a table to place the clothes on the table and folding them, or folding the clothes without using a table. In this case, the judgment unit 402 determines whether or not the user uses a table when folding clothes. For example, the judgment unit 402 determines the presence or absence of a table around the user from the surrounding image captured by the camera. Then, the judgment unit 402 determines that if there is a table nearby, the user uses a table when folding clothes. Also, the judgment unit 402 determines that if there is no table nearby, the user does not use a table when folding clothes. The judgment unit 402 may then set the criteria for judgment according to the determined presence or absence of a table. The judgment unit 402 determines the presence or absence of a table, for example, from the user's surrounding image as described above, or from a sensor that measures the distance to surrounding objects.

[0111] Furthermore, the judgment criteria may be set according to actions performed in conjunction with the action of folding clothes. The judgment criteria for each action may be set in the same manner as in the first embodiment. The judgment unit 402 identifies each action in the following situations detected by the detection unit 401. Then, the judgment unit 402 determines the correctness of the action based on the judgment criteria corresponding to the identified action.

[0112] For example, the scenario is one in which a cashier is assisting a customer at a register. In this scenario, the detection unit 401 detects the user folding clothes as well as the payment process. Alternatively, in this scenario, the detection unit 401 may also detect at least one of the following actions: the user bagging items, or engaging in conversation with the customer.

[0113] For example, the detection unit 401 detects actions in the accounting process where the user receives goods, cash, gift certificates, credit cards or other cards used for payment, or point cards from the customer. The detection unit 401 also detects actions in the accounting process where the user hands over goods, change, cards, receipts, promotional papers, etc. to the customer. Furthermore, the detection unit 401 detects actions in the accounting process where the user operates the cash register. For example, cash register operations include at least one of the following: operating the cash register for various payment methods, registering goods at the cash register, canceling goods, adding points, or subtracting points. Cash register operations are not limited to these. The determination unit 402 determines the correctness of each operation based on the determination criteria corresponding to the operation detected by the detection unit 401.

[0114] Furthermore, when the detection unit 401 detects a conversation between a user and a customer, it acquires, for example, the voice spoken by the user. The detection unit 401 then detects the words spoken by the user from the voice spoken by the user. The determination unit 402 determines the correctness of the user's utterances in the conversation with the customer based on the words detected by the detection unit 401 and the determination criteria. The determination criteria for user utterances may be the content of the user's utterances and the wording used by the user. In determining each action, the determination unit 402 may also determine the correctness of the action and the utterance.

[0115] Alternatively, the situation is one in which a user is organizing shelves in a store. In this situation, the user performs at least one of the following actions in addition to folding clothes: placing clothes on a shelf, or organizing clothes on a shelf. In this situation, the user may also rearrange the shelves or change the price tags on the products.

[0116] Alternatively, the situation is one in which clothes are folded in the back room of a store. In the situation of folding clothes in the back room of a store, the user may, in addition to folding clothes, perform at least one of the following actions: returning clothes, packing clothes into a box or bag, or altering clothes such as hemming.

[0117] As described above, the detection unit 401 may detect not only the user's actions of folding clothes, but also other actions performed by the user in the store, as well as the user's speech directed at customers.

[0118] Furthermore, the output unit 403 outputs the determination result to the output device, similar to the embodiments described above.

[0119] In the folding method evaluation device 400 of this embodiment, when a user folds a virtual garment, the folding method evaluation device 400 includes a display control unit 404.

[0120] The display control unit 404 displays a virtual space according to each situation. For example, in a situation where there is a table, the display control unit 404 displays a virtual table as shown in Figures 9 and 10. In this case, the display control unit 404 also displays the virtual clothing as if it were placed on the virtual table. In a situation where there is no table, the display control unit 404 does not display a virtual table, as shown in the display example in Figure 20.

[0121] Furthermore, in a scenario where the user is assisting a customer at a cash register, the display control unit 404 displays a virtual cash register and counter, as well as a virtual customer. In this case, the display control unit may also display a virtual shopping basket and a virtual bag. Figure 21 shows an example of a display by the display control unit 404 showing virtual clothing, a virtual cash register, a virtual counter, and a virtual customer. Figure 22 shows an example of a display by the display control unit 404 showing bag options, virtual clothing, a virtual cash register, a virtual counter, and a virtual customer.

[0122] Furthermore, when the display control unit 404 displays a virtual customer, the display control unit 404 may display the customer's utterances as text. Alternatively, the display control unit 404 may display a selection of user utterances to the customer. In this case, the determination unit 402 should determine the correctness of the utterance selection made by the user based on a determination criterion that associates the utterance selections with correct answer information. Or, when the display control unit 404 displays a virtual customer, an audio control unit (not shown) may output the customer's utterances as audio. The audio control unit outputs the audio to an audio output device such as a speaker, earphones, or headphones that the user can hear. For example, the audio control unit may output audio of the customer's utterances at an output timing associated with the customer's utterances. Alternatively, the audio control unit may convert the text of the customer's utterances into text-to-speech audio and output the audio at an output timing associated with the customer's utterances.

[0123] In the folding method evaluation device of the embodiment described above, the detection unit detects the user's actions of folding clothes based on sensor information. The detection unit also detects each action according to the situation. The determination unit determines the correctness of the detected actions based on determination criteria corresponding to the situation and the clothes. The output unit outputs the determination result to the output device.

[0124] As a result, the folding method evaluation device in this embodiment can support the learning of how to fold clothes. In particular, the folding method evaluation device in this embodiment can support the evaluation of learning various tasks in a store, in addition to how to fold clothes.

[0125] [Hardware configuration] Some or all of the components of each apparatus or system in each embodiment of the present disclosure described above can be realized by any combination of an information processing device 1000 and a program, for example, as shown in Figure 23. The information processing device 1000 includes, as an example, the following configuration.

[0126] ·CPU(Central Processing Unit)1001 • ROM (Read Only Memory) 1002 ·RAM(Random Access Memory)1003 • Program 1004 loaded into RAM1003 • Storage device 1005 for storing program 1004 • Drive device 1007 for reading and writing recording medium 1006 • Communication interface 1008 connected to communication network 1009 • Input / output interface (I / F) 1010 for data input and output. • Bus 1011 connecting each component

[0127] Each component of each device or system in each embodiment is realized by the CPU 1001 acquiring and executing a program that realizes its function. The program that realizes the function of each component of each device is, for example, stored in advance in the storage device 1005 or RAM 1003 and read by the CPU 1001 as needed. The program 1004 may be supplied to the CPU 1001 via a communication network, or it may be stored in advance in the recording medium 1006 and read by the drive device 1007 and supplied to the CPU 1001.

[0128] There are various variations in how each device is implemented. For example, each device or system may be implemented by any combination of a separate information processing device 1000 and a program for each component. Alternatively, the multiple components of each device may be implemented by any combination of a single information processing device 1000 and a program.

[0129] Furthermore, some or all of the components of each device or system are realized by general-purpose or dedicated circuits, including processors, or combinations thereof. Examples of circuits include CPUs, GPUs (Graphics Processing Units), FPGAs (Field Programmable Gate Arrays), and LSIs (Large Scale Integrations). An LSI is, for example, a dedicated LSI for AI (Artificial Intelligence) processing. These may consist of a single chip or multiple chips connected via a bus. Some or all of the components of each device may be realized by combinations of the aforementioned circuits and programs.

[0130] When some or all of the components of each device or system are implemented by multiple information processing devices or circuits, these multiple information processing devices or circuits may be centrally located or distributed. For example, the information processing devices or circuits may be implemented in a form in which each is connected via a communication network, such as a client-and-server system or a cloud computing system.

[0131] Although the present invention has been described above with reference to the embodiments described, the present invention is not limited to the above embodiments. Various modifications to the configuration and details of the present invention can be made that will be understood by those skilled in the art within the scope of the present invention.

[0132] Furthermore, although multiple operations are described sequentially in flowchart format, the order in which they are described does not limit the order in which they must be performed. Therefore, when implementing each embodiment, the order of the multiple operations may be changed to the extent that it does not impair the content. [Explanation of Symbols]

[0133] 10. Folding Method Evaluation System 100 Folding method evaluation device 101 Detection unit 102 Judgment section 103 Output section 200 Folding Method Evaluation Device 201 Detection Unit 202 Judgment section 203 Output section 204 Display Control Unit 300 Folding Method Evaluation Device 301 Detection unit 302 Judgment section 303 Output section 400 Folding Method Evaluation Device 401 Detection Unit 402 Judgment section 403 Output section 1000 Information Processing Devices 1001 CPU 1002 ROM 1003 RAM 1004 Program 1005 Storage device 1006 Recording media 1007 Drive unit 1008 Communication I / F 1009 Communication Network 1010 Input / Output Interface 1011 Bus

Claims

1. A detection means for detecting the user's actions based on sensor information, which is an image taken from a camera of the user folding clothes, or information measuring the user's actions taken from a motion sensor attached to the user's body. A determination means for determining the correctness of the detected action based on determination criteria obtained from a storage unit according to the type of clothing entered by the user or the user's administrator, or the type of clothing estimated from the image captured by the camera, An output means for outputting the judgment result to an output device, A folding method evaluation device equipped with [a specific feature].

2. The detection means further detects the state of the clothes that the user is folding from the image, The determination means further determines whether the state of the clothing is correct in relation to the action. The folding method evaluation device according to claim 1.

3. The system further includes a display control means for displaying virtual clothing folded by the user in a virtual space on a head-mounted display. The folding method evaluation device according to claim 1.

4. The display control means changes the state of the virtual clothing to be displayed in accordance with the detected action. The folding method evaluation device according to claim 3.

5. The determination means determines the user's action based on the type of clothing and the determination criteria corresponding to the situation in which the clothing is folded. The folding method evaluation device according to claim 1.

6. The determination means determines the action taken by the user based on the determination criteria corresponding to the situation, which are based on the user's surrounding image. The folding method evaluation device according to claim 5.

7. The determination means determines the user's action based on the determination criteria corresponding to the situation in which there is a table for folding clothes or in which there is no such table. The folding method evaluation device according to claim 5 or 6.

8. The determination means determines the user's actions based on the determination criteria corresponding to the customer service situation at the register. A folding method evaluation device according to claim 5 or 6.

9. The aforementioned customer service includes accounting procedures and conversations with customers. The detection means further detects the accounting operation and the voice spoken by the user, The determination means further determines the correctness of the detected accounting operation and voice. The folding method evaluation device according to claim 8.

10. The detection means further detects the bagging action performed by the user, The determination means further determines the correctness of the detected bagging operation. The folding method evaluation device according to claim 1.

11. The determination means determines whether the bag selected by the user from among the detected bagging operations is appropriate for the product to be bagged. The folding method evaluation device according to claim 10.

12. The determination means determines the correctness of the location where the clothes are held among the detected actions. The folding method evaluation device according to claim 1.

13. The determination means determines whether the sequence of the actions of folding clothes is correct among the detected actions. The folding method evaluation device according to claim 1.

14. The determination means determines the correctness of the operation for each of the multiple steps based on the determination criteria set for each of the multiple steps of the operation. The folding method evaluation device according to claim 1.

15. The output means outputs information to the output device indicating that the operation is incorrect as a result of the determination. The folding method evaluation device according to claim 1.

16. Computers Based on sensor information, which is either an image taken from a camera capturing the user folding clothes, or information measuring the user's actions taken from a motion sensor attached to the user's body, the system detects the user's actions. Based on the type of clothing entered by the user or the user's administrator, or the type of clothing estimated from the image captured by the camera, the system determines the correctness of the action of folding the detected clothing according to the criteria obtained from the storage unit. Output the judgment result to the output device. Method for evaluating folding techniques.

17. A process for detecting the user's actions based on sensor information, which is an image taken from a camera of the user folding clothes, or information measuring the user's actions taken from a motion sensor attached to the user's body. A process to determine the correctness of the action of folding the detected clothing based on the type of clothing entered by the user or the user's administrator, or the type of clothing estimated from the image captured by the camera, and judgment criteria obtained from the storage unit. The process of outputting the judgment result to the output device, A program that uses a computer to evaluate folding methods.

Images