Human-machine interaction device and control method thereof

Abstract

This invention provides a control method for a human-computer interaction device suitable for refrigerator detection. The device includes an information output module and a detection module. The control method includes: controlling the information output module to output operation commands so that an operator can operate the refrigerator according to the commands; controlling the detection module to detect whether the operator has performed a corresponding operation on the refrigerator; and responding to the detection that the operator has performed a corresponding operation, controlling the information output module to output the operation commands again. This human-computer interaction device ensures that all functional components and functions of each refrigerator are detected by providing a series of operation commands to the operator.

Description

Technical Field

[0001] This invention belongs to the field of human-computer interaction technology, and specifically provides a human-computer interaction device and its control method. Background Technology

[0002] During the refrigerator manufacturing process, it is often necessary to test various functional components and functions of the refrigerator. Currently, the testing of some functional components and functions of refrigerators often requires manual intervention, that is, operators test certain functional components and functions of the refrigerator.

[0003] To ensure that operators are competent in inspecting refrigerators, extensive training is required before they can begin their work, consuming significant human and financial resources. Furthermore, since everyone is prone to error, even skilled operators may miss items during inspection, potentially leading to defective or poorly designed products entering the market. Summary of the Invention

[0004] One object of the present invention is to provide a human-computer interaction device to guide operators in completing the inspection of a refrigerator and to prevent operators from missing inspections.

[0005] To achieve the above objectives, the present invention provides, in a first aspect, a control method for a human-computer interaction device, the human-computer interaction device being suitable for refrigerator detection, the human-computer interaction device comprising an information output module and a detection module; the control method comprising:

[0006] The information output module is controlled to output operation instructions so that the operator can operate the refrigerator according to the operation instructions.

[0007] The detection module is controlled to detect whether the operator has performed any corresponding operations on the refrigerator;

[0008] In response to the detection that the operator has performed the corresponding operation, the information output module is controlled to output the operation command again.

[0009] Optionally, the human-computer interaction device further includes a model reading module, and before the information output module outputs operation instructions, the control method further includes:

[0010] The model reading module is controlled to read the model information of the refrigerator;

[0011] Each of the model information corresponds to a set of operation instructions.

[0012] Optionally, the model reading module is a barcode scanning device.

[0013] Optionally, the detection module is communicatively connected to the refrigerator, and controlling the detection module to detect whether an operator has performed a corresponding operation on the refrigerator includes:

[0014] The detection module is controlled to detect whether the refrigerator has issued information corresponding to the operation command, so as to determine whether the operator has performed the corresponding operation on the refrigerator.

[0015] Optionally, the human-computer interaction device further includes an indicator module configured to emit a light beam;

[0016] After the information output module outputs the operation command, the control method further includes:

[0017] The indicator module is controlled to direct the light beam to the part of the refrigerator to be operated, so as to guide the operator to find the part to be operated.

[0018] Optionally, the human-computer interaction device further includes an image acquisition module.

[0019] The control method further includes:

[0020] The image acquisition module is controlled to acquire images of the refrigerator;

[0021] The location of the part to be operated on is determined from the image;

[0022] The control module for directing the beam of light to the area of ​​the refrigerator to be operated includes:

[0023] Based on the position of the part to be operated in the image, the indicator module is controlled to direct the light beam to the part to be operated on the refrigerator.

[0024] Optionally, the human-computer interaction device includes multiple image acquisition modules.

[0025] Determining the location of the part to be operated on from the image includes:

[0026] Each image acquired by the image acquisition module is checked to determine whether the part to be operated on exists in the image, so as to identify at least one image containing the part to be operated on.

[0027] The location of the part to be operated is determined from at least one of the images.

[0028] Optionally, the human-computer interaction device includes multiple indicator modules.

[0029] The step of controlling the indicator module to direct the light beam onto the refrigerator's target area based on the location of the target area in the image includes:

[0030] Based on the position of the part to be operated in the image, at least one indicator module capable of directing a light beam to the part to be operated on the refrigerator is determined;

[0031] The at least one of the indicated modules is controlled to direct the light beam onto the part of the refrigerator to be operated.

[0032] Optionally, the information output module includes, but is not limited to, a display screen and / or a speaker, wherein the display screen is used to display the operation instructions and the speaker is used to broadcast the operation instructions.

[0033] The present invention provides a human-computer interaction device in a second aspect, the human-computer interaction device being the human-computer interaction device described in any one of the first aspects, the human-computer interaction device further comprising a control module, a storage module, and execution instructions stored in the storage module, the execution instructions being configured to enable the human-computer interaction device to perform any one of the control methods in the first aspect when executed by the control module.

[0034] Based on the foregoing description, those skilled in the art will understand that in the aforementioned technical solution of this invention, the operation instructions are output by the control information output module, enabling the operator to operate the refrigerator according to the instructions, thus avoiding situations where the operator misses any functional modules or functions of the refrigerator. The control detection module detects whether the operator has performed the corresponding operation on the refrigerator, allowing the human-machine interface device of this invention to check the operator's operation results, preventing operator errors or improper operation, thereby ensuring the quality and appearance of the refrigerator. After detecting that the operator has performed the corresponding operation, the control information output module outputs the operation instructions again, enabling the human-machine interface device of this invention to provide the operator with a series of operation instructions, thereby ensuring that all functional components and functions of each refrigerator are detected.

[0035] Furthermore, the refrigerator model information is read by the model reading module, and then the information output module outputs different operation instructions according to different model information. This allows the human-computer interaction device of the present invention to adjust the output operation instructions according to the refrigerator model, thereby ensuring that each model of refrigerator can be quickly and accurately tested by the operator.

[0036] Furthermore, by establishing a communication connection between the detection module and the refrigerator, the detection module can determine whether the operator has performed the corresponding operation on the refrigerator by detecting whether the refrigerator has issued information corresponding to the operation command.

[0037] Furthermore, the indicator module directs the light beam onto the area of ​​the refrigerator to be operated, guiding the operator to locate it. This allows operators, especially those with limited experience, to quickly find and inspect the area.

[0038] The above and other objects, advantages and features of the present invention will become more apparent to those skilled in the art from the following detailed description of specific embodiments of the invention in conjunction with the accompanying drawings. Attached Figure Description

[0039] To more clearly illustrate the technical solution of the present invention, some embodiments of the present invention will be described below with reference to the accompanying drawings. Those skilled in the art should understand that the same reference numerals may indicate the same or similar parts or components in different drawings; the drawings of the present invention are not necessarily drawn to scale.

[0040] In the attached image:

[0041] Figure 1 This is a schematic diagram illustrating the usage effect of the human-computer interaction device in this invention;

[0042] Figure 2 These are schematic diagrams illustrating the structure of human-computer interaction devices in some embodiments of the present invention;

[0043] Figure 3 This is a flowchart of the main steps of the control method for a human-computer interaction device in some embodiments of the present invention;

[0044] Figure 4 These are schematic diagrams illustrating the configuration of the human-computer interaction device in other embodiments of the present invention;

[0045] Figure 5 This is a schematic diagram of the human-computer interaction device in some embodiments of the present invention;

[0046] Figure 6 This is a flowchart of some steps of the control method for a human-computer interaction device in some embodiments of the present invention. Detailed Implementation

[0047] Those skilled in the art should understand that the embodiments described below are merely a part of the embodiments of the present invention, and not all of the embodiments of the present invention. These partial embodiments are intended to explain the technical principles of the present invention and are not intended to limit the scope of protection of the present invention. Based on the embodiments provided by the present invention, all other embodiments obtained by those skilled in the art without creative effort should still fall within the scope of protection of the present invention.

[0048] It should be noted that in the description of this invention, terms such as "center," "upper," "lower," "top," "bottom," "left," "right," "vertical," "horizontal," "inner," and "outer," which indicate direction or positional relationships, are based on the direction or positional relationships shown in the accompanying drawings. These are used merely for ease of description and do not indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation; therefore, they should not be construed as limitations on this invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0049] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can also refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0050] Furthermore, it should be noted that in the description of this invention, each functional module can be a physical module composed of multiple structures, components, or electronic devices, or a virtual module composed of multiple programs; each functional module can be an independent module or a module divided from a whole module according to its function. Those skilled in the art should understand that, provided the technical solution described in this invention can be implemented, any changes in the configuration, implementation, or positional relationship of the functional modules will not deviate from the technical principles of this invention, and therefore should all fall within the protection scope of this invention.

[0051] like Figure 1 As shown, the human-computer interaction device 100 of the present invention is applicable to the auxiliary inspection of a refrigerator 200. Specifically, the human-computer interaction device 100 of the present invention can assist operators in inspecting defects in the refrigerator door, the display performance of the screen, the drawer pull-out effect, and whether the ice-making module's piping is leaking, etc. Those skilled in the art can add or remove items that the human-computer interaction device 100 assists operators in inspecting the refrigerator 200 as needed.

[0052] The refrigerator 200 being inspected is located on the conveyor line 300 so that it can be moved away by the conveyor line 300 after it has been inspected, and the human-machine interface device 100 assists the operator in inspecting the next refrigerator 200.

[0053] It should be noted that the human-computer interaction device 100 of the present invention can also be applied to the auxiliary detection of any other feasible device, such as an air conditioner, a washing machine, etc.

[0054] The following reference Figure 2 and Figure 3 This will be explained in detail regarding the structure and control method of the human-computer interaction device 100 in some embodiments of the present invention. Figure 2 These are schematic diagrams illustrating the configuration of human-computer interaction devices in some embodiments of the present invention. Figure 3 This is a flowchart of the main steps of the control method for a human-computer interaction device in some embodiments of the present invention.

[0055] like Figure 2 As shown, the human-computer interaction device 100 includes a control module 110, an information output module 120, and a detection module 130. The control module 110 is communicatively connected to the information output module 120 and the detection module 130, respectively.

[0056] It should be noted that, in this invention, "communication connection" can refer to connecting two or more corresponding entities together via a conductor, or it can refer to connecting two or more corresponding entities together via wireless signals (such as WiFi, Bluetooth, ZigBee, 4G, 5G, etc.).

[0057] In some embodiments of the present invention, the control module 110 is at least used to control other functional modules of the human-machine interface device 100 to perform corresponding operations. The control module 110 can be any feasible module, such as a CPU, a PID controller, a PLC controller, etc.

[0058] In some embodiments of the present invention, the information output module 120 is used to output operation instructions, which may be sound, graphics, and / or text. The information output module 120 includes, but is not limited to, at least one of a display screen and a speaker. When the information output module 120 includes a display screen, the display screen is used to display text and / or graphics to output operation instructions in the form of the text and / or graphics. When the information output module 120 includes a speaker, the speaker is used to output sound to output operation instructions in the form of the sound.

[0059] In some embodiments of the present invention, the detection module 130 is communicatively connected to the refrigerator 200; specifically, the detection module 130 is communicatively connected to the controller of the refrigerator 200. The detection module 130 determines the operating status of each functional component of the refrigerator 200 by receiving information from the controller of the refrigerator 200.

[0060] Of course, those skilled in the art can also, as needed, make the detection module 130 communicate only with the part of the refrigerator 200 to be detected. For example, when the refrigerator 200 needs to detect whether the refrigerator door opens and closes normally or whether the drawer pushes and pulls normally, the detection module 130 can communicate with the door switch sensor and drawer sensor on the refrigerator 200.

[0061] In addition, those skilled in the art can, as needed, make the detection module 130 include a camera module so that the detection module 130 can capture the posture changes of the components on the refrigerator 200 through the camera module, thereby determining whether the operator has performed the corresponding operation through the posture changes.

[0062] The following reference Figure 3 The control methods of the human-computer interaction device 100 in some embodiments of the present invention will be described in detail below.

[0063] like Figure 3 As shown, in some embodiments of the present invention, the control method of the human-computer interaction device 100 includes:

[0064] Step S110 causes the control module 110 to output an operation command to the control information output module 120, so that the operator can operate the refrigerator 200 according to the operation command.

[0065] In step S120, the control module 110 controls the detection module 130 to detect whether the operator has performed the corresponding operation on the refrigerator 200.

[0066] Specifically, the detection module 130 is made communicatively connected to the controller of the refrigerator 200. If the detection module 130 receives the target information fed back by the controller of the refrigerator 200, it determines that the operator has performed the corresponding operation on the refrigerator 200. If the detection module 130 does not receive the target information fed back by the controller of the refrigerator 200, it determines that the operator has not performed the corresponding operation on the refrigerator 200.

[0067] The target information corresponds to the operation instructions output by the information output module 120.

[0068] In step S130, in response to detecting that the operator has performed the corresponding operation, the control module 110 control information output module 120 outputs the operation command again.

[0069] Specifically, when the operator performs the corresponding operation on the refrigerator 200, the control module 110 and the control information output module 120 output the operation command again.

[0070] Those skilled in the art will understand that, because the components or functions to be tested by the refrigerator 200 are usually different each time, the operation commands output by the information output module 120 are different each time. Examples are given below:

[0071] The refrigerator 200 requires manual inspection for the following items: whether the display screen is functioning correctly, whether the refrigerator door opens and closes properly, and whether the drawers slide open and close properly. Assuming the inspection order for these three items is: whether the display screen is functioning correctly, whether the refrigerator door opens and closes properly, and whether the drawers slide open and close properly, then the control method for the human-machine interface device 100 is as follows:

[0072] First, the control module 110 and control information output module 120 output a first operation command, which includes information indicating "checking whether the display screen is displaying normally". This allows the operator to check whether the display screen is displaying normally according to the first operation command.

[0073] During the operator's inspection of the display screen to ensure it is functioning correctly, the refrigerator 200's controller will control the display screen to show the corresponding content. After detecting the controller's instruction to control the display screen, the detection module 130 determines that the operator has completed the display screen inspection.

[0074] Then, the control module 110 controls the information output module 120 to output a second operation command, which includes information indicating "checking whether the refrigerator door opens and closes normally". This allows the operator to check whether the refrigerator door opens and closes normally according to the second operation command.

[0075] During the operator's inspection of the refrigerator door's opening and closing mechanism, opening the door triggers the door switch sensor on the refrigerator, which detects door opening and closing. When this sensor is triggered, it sends corresponding information to the refrigerator 200's controller. Once the detection module 130 detects that the controller has received this information, it determines that the operator has completed the refrigerator door inspection.

[0076] Finally, the control module 110 and control information output module 120 output a third operation instruction, which includes information indicating "checking whether the drawer's push-pull mechanism is functioning correctly." This allows the operator to check whether the drawer's push-pull mechanism is functioning correctly based on the third operation instruction.

[0077] During the operator's check of whether the drawer's push and pull functions are normal, the pulled drawer will trigger the drawer sensor on the refrigerator 200. When the drawer sensor is triggered, it sends corresponding information to the controller of the refrigerator 200. When the detection module 130 detects that the controller has received this information, it determines that the operator has completed the drawer check.

[0078] Based on the foregoing description, those skilled in the art will understand that in some embodiments of the present invention, the control information output module 120 outputs operation instructions, enabling the operator to operate the refrigerator 200 according to the operation instructions, thus avoiding situations where the operator misses any functional modules or functions of the refrigerator 200. The control detection module 130 detects whether the operator has performed the corresponding operation on the refrigerator 200, enabling the human-machine interface device 100 of the present invention to check the operator's operation results, avoiding operator errors or improper operation, thereby ensuring the quality and appearance of the refrigerator 200. After detecting that the operator has performed the corresponding operation, the control information output module 120 outputs operation instructions again, enabling the human-machine interface device 100 of the present invention to provide the operator with a series of operation instructions, thereby ensuring that all functional components and functions of each refrigerator 200 are detected.

[0079] The following reference Figure 4 The following provides a detailed description of the configuration and control method of the human-computer interaction device 100 in other embodiments of the present invention.

[0080] It should be noted beforehand that, for ease of description and to enable those skilled in the art to quickly understand the technical solutions of the present invention, the following description will only detail the differences between some embodiments and the foregoing embodiments. Similarities between the other embodiments and the foregoing embodiments can be found above.

[0081] like Figure 4 As shown, in some other embodiments of the present invention, the human-computer interaction device 100 further includes a model reading module 140, which is communicatively connected to the control module 110. The model reading module 140 is used to read the model information of the refrigerator 200.

[0082] In other embodiments of the present invention, the model reading module 140 can be any feasible module, such as a scanning device, a camera device, an RFID receiver, etc. Accordingly, the refrigerator 200 is provided with components that can be read by the model reading module 140. For example, when the model reading module 140 is a scanning device and / or a camera device, the refrigerator 200 is provided with a graphic code (e.g., a one-dimensional barcode and a QR code). When the model reading module 140 is an RFID reader / writer, the refrigerator 200 is provided with an RFID card.

[0083] Furthermore, in some other embodiments of the present invention, the control method of the human-computer interaction device 100 further includes a step prior to step S110: causing the control module 110 to control the model reading module 140 to read the model information of the refrigerator 200, so as to determine a set of operation instructions based on the model information of the refrigerator 200, and then causing the information output module 120 to output each operation instruction in the set of operation instructions sequentially.

[0084] Preferably, the control module 110, or a storage module or server communicatively connected to the control module 110, stores multiple model information entries, and each model information entry corresponds to a set of operation instructions. When the model reading module 140 reads the model information of the refrigerator 200, the control module 110 matches the model information read by the model reading module 140 with the stored multiple model information entries. Then, based on the matching result, it determines a unique model information entry and a set of operation instructions.

[0085] Although not shown in the figure, the model reading module 140 is preferably set on one side of the conveyor line 300. When the refrigerator 200 moves with the conveyor line 300 to a position aligned with the model reading module 140, the model reading module 140 reads the model information of the refrigerator 200.

[0086] Based on the foregoing description, those skilled in the art will understand that in other embodiments of the present invention, the model reading module 140 reads the model information of the refrigerator 200, and then the information output module 120 outputs different operation instructions according to different model information, so that the human-computer interaction device 100 of the present invention can adjust the output operation instructions according to the model of the refrigerator 200, thereby ensuring that each model of refrigerator 200 can be quickly and accurately tested by the operator.

[0087] The following reference Figure 5 and Figure 6 The following provides a detailed description of the configuration and control method of the human-computer interaction device 100 in some embodiments of the present invention. Figure 5 This is a schematic diagram illustrating the configuration of the human-computer interaction device 100 in some embodiments of the present invention. Figure 6 This is a flowchart of some steps of the control method of the human-computer interaction device 100 in some embodiments of the present invention.

[0088] It should be noted beforehand that, for ease of description and to enable those skilled in the art to quickly understand the technical solutions of the present invention, the following description will only detail the differences between some embodiments and some or other embodiments described above. Similarities between some embodiments and some of the foregoing embodiments can be found above.

[0089] like Figure 5 As shown, in some embodiments of the present invention, the human-computer interaction device 100 further includes an indication module 150 and an image acquisition module 160. The indication module 150 and the image acquisition module 160 are respectively communicatively connected to the control module 110.

[0090] The indicator module 150 is configured to emit a light beam, which is directed onto the area of ​​the refrigerator 200 to be operated, allowing the operator to quickly locate that area. This area is the part that requires operation or inspection by the operator, such as the refrigerator 200's display screen, door, or rear panel.

[0091] In some embodiments of the present invention, the indicating module 150 is preferably a laser module, so that the light beam emitted by the indicating module 150 is a laser beam. Those skilled in the art will understand that, compared to a regular light beam, a laser beam is less prone to divergence and is more easily projected onto a single point, which is beneficial for locating small parts to be operated on. Furthermore, those skilled in the art can also, as needed, configure the indicating module 150 as any other feasible light-emitting module, such as a spotlight.

[0092] More preferably, there are multiple indicator modules 150. These multiple indicator modules 150 are distributed such that at least one indicator module 150 can direct the light beam to the part of the refrigerator 200 to be operated. This avoids the situation where a single indicator module 150 is blocked by other parts of the refrigerator 200 (such as the side wall or partition of the refrigerator 200), thus preventing it from directing the light beam to the part of the refrigerator 200 to be operated.

[0093] Optionally, the plurality of indicator modules 150 are distributed at intervals on the side of the human-computer interaction device 100 facing the refrigerator 200.

[0094] The image acquisition module 160 is used to acquire images of the refrigerator 200. Preferably, the human-computer interaction device 100 includes multiple image acquisition modules 160, so that at least one of the multiple image acquisition modules 160 can capture images of the operable parts of the refrigerator 200. Optionally, the multiple image acquisition modules 160 are distributed at intervals on the side of the human-computer interaction device 100 facing the refrigerator 200.

[0095] In some other embodiments of the present invention, the image acquisition module 160 can be any feasible module, such as a camera, camcorder, or video camera.

[0096] like Figure 6 As shown, in some other embodiments of the present invention, the control method of the human-computer interaction device 100 further includes:

[0097] Step S310: The control module 110 controls the image acquisition module 160 to acquire images of the refrigerator 200.

[0098] Step S310 can be performed before or after step S110, or step S310 can be performed simultaneously with step S110.

[0099] Preferably, the control module 110 controls each image acquisition module 160 to acquire images of the refrigerator 200.

[0100] Step S320, located between steps S110 and S120, enables the control module 110 to determine the position of the part to be operated from the image.

[0101] Step S320 further includes:

[0102] Step S321: The control module 110 acquires all the images and detects whether there is a part to be operated in each image acquired by the image acquisition module 160, so that the control module 110 determines that there is at least one image with a part to be operated.

[0103] Step S322: The control module 110 determines the location of the part to be operated from the at least one image.

[0104] Specifically, the control module 110 randomly selects an image containing the part to be operated on, and then determines the position of the part to be operated on from that image.

[0105] In step S330, which is located between step S110 and step S120, the control module 110 controls the indicator module 150 to direct the beam of light onto the part of the refrigerator 200 to be operated, based on the position of the part to be operated in the image.

[0106] Specifically, the control module 110 determines the world coordinates of the part to be operated on the refrigerator 200 based on the position of the part to be operated in the image and the world coordinates, image coordinate system, and camera coordinate system of the corresponding image acquisition module 160. Then, the control module 110 determines the positional relationship between the indicator module 150 and the part to be operated based on the world coordinates of the indicator module 150 and the part to be operated. Finally, the control module 110 controls the corresponding indicator module 150 to project a light beam onto the part to be operated on the refrigerator 200 based on this positional relationship.

[0107] Based on the foregoing description, those skilled in the art will understand that in some embodiments of the present invention, the indicator module 150 directs the light beam to the part of the refrigerator 200 to be operated, thereby guiding the operator to find the part to be operated, so that the operator, especially the less experienced operator, can quickly find the part to be operated and inspect it.

[0108] Furthermore, in some other embodiments of the present invention, the human-computer interaction device 100 may also include a storage module communicatively connected to the control module 110. This storage module stores execution instructions configured to enable the human-computer interaction device 100 to perform the control method described in any of the foregoing embodiments when executed by the control module 110.

[0109] The memory stores execution instructions, specifically executable computer programs. Further, the memory may include main memory and non-volatile memory, providing execution instructions and data to the processor. For example, main memory may be high-speed random-access memory (RAM), and non-volatile memory may be at least one disk storage device.

[0110] The technical solutions of the present invention have been described in conjunction with several embodiments above. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is not limited to these specific embodiments. Without departing from the technical principles of the present invention, those skilled in the art can disassemble and combine the technical solutions in the above embodiments, and can also make equivalent changes or substitutions to related technical features. Any changes, equivalent substitutions, improvements, etc., made within the technical concept and / or technical principles of the present invention will fall within the scope of protection of the present invention.

Claims

1. A control method for a human-computer interaction device, the human-computer interaction device being suitable for detecting refrigerators, the human-computer interaction device comprising an information output module, a detection module, an indicator module, and an image acquisition module, the indicator module being configured to emit a light beam; The control method includes: The information output module is controlled to output operation instructions so that the operator can operate the refrigerator according to the operation instructions. The image acquisition module is controlled to acquire images of the refrigerator; The location of the part of the refrigerator to be operated is determined from the image; Based on the location of the part to be operated in the image, the indicator module is controlled to direct the beam of light to the part to be operated on the refrigerator, so as to guide the operator to quickly find the part to be operated. The detection module is controlled to detect whether the operator has performed any corresponding operations on the refrigerator; In response to the detection that the operator has performed the corresponding operation, the information output module is controlled to output the operation command again.

2. The control method for the human-computer interaction device according to claim 1, wherein, The human-computer interaction device also includes a model reading module. Before the information output module outputs the operation command, the control method further includes: The model reading module is controlled to read the model information of the refrigerator; Each of the model information corresponds to a set of operation instructions.

3. The control method for the human-computer interaction device according to claim 2, wherein, The model reading module is a barcode scanning device.

4. The control method for the human-computer interaction device according to claim 1, wherein, The detection module is communicatively connected to the refrigerator. The control module detects whether the operator has performed a corresponding operation on the refrigerator, including: The detection module is controlled to detect whether the refrigerator has issued information corresponding to the operation command, so as to determine whether the operator has performed the corresponding operation on the refrigerator.

5. The control method for the human-computer interaction device according to claim 1, wherein, The human-computer interaction device includes multiple image acquisition modules. Determining the location of the part to be operated on from the image includes: Each image acquired by the image acquisition module is checked to determine whether the part to be operated on exists in the image, so as to identify at least one image containing the part to be operated on. The location of the part to be operated is determined from at least one of the images.

6. The control method for the human-computer interaction device according to claim 1 or 5, wherein, The human-computer interaction device includes multiple of the aforementioned indicator modules. The step of controlling the indicator module to direct the light beam onto the refrigerator's target area based on the location of the target area in the image includes: Based on the position of the part to be operated in the image, at least one indicator module capable of directing a light beam to the part to be operated on the refrigerator is determined; The at least one of the indicated modules is controlled to direct the light beam onto the part of the refrigerator to be operated.

7. The control method for the human-computer interaction device according to any one of claims 1 to 5, wherein, The information output module includes, but is not limited to, a display screen and / or a speaker. The display screen is used to display the operation instructions. The speaker is used to broadcast the operation instructions.

8. A human-computer interaction device, wherein the human-computer interaction device is the human-computer interaction device according to any one of claims 1 to 7. The human-computer interaction device further includes a control module, a storage module, and execution instructions stored on the storage module. The execution instructions are configured to enable the human-computer interaction device to perform the control method according to any one of claims 1 to 7 when executed by the control module.

Images