How to Recognize Combinations of Objects in Augmented Reality Images

Abstract

To provide a method for recognizing at least one object in an image when performing interaction of augmented reality using a camera and a processor.SOLUTION: At least one object includes content information and operation direction information. The operation direction information includes an inspection direction, a re-inspection direction, and a combination thereof. The method includes a first re-inspection step, and the first re-inspection step includes using the processor to inspect whether one inspection direction of the at least one object and any one re-inspection direction of its adjacent at least one object satisfy a pairing relationship. If so, the processor processes content information of either the at least one object or another one object to generate a computational result.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a method for stably combining two or more objects in an image during an augmented reality interaction.

[0002] With the evolution of augmented reality technology, more and more application scenarios are being developed and utilized. Among them, augmented reality card games have become a new form of interaction method, which usually uses a mobile device that supports augmented reality function to detect the patterns of cards and provide corresponding augmented reality information. However, in the conventional technology, when the camera of the mobile device detects a combination of two or more card patterns at the same time, the detection may be unstable. Summary of the Invention [Means for solving the problem]

[0003] An embodiment of the present invention provides a method for recognizing at least one object in an image using a camera and a processor, the at least one object including content information and computation direction information, the computation direction information including an inspection direction, a re-inspection direction, or a combination thereof. The method includes a first re-inspection step, the first re-inspection step including inspecting by the processor whether the inspection direction of one of the at least one object and the re-inspection direction of any one of its adjacent at least one object satisfy a pairing relationship. If the pairing relationship is satisfied, the processor processes the content information of the at least one object and any one of the at least one object to generate a computation result.

[0004] An embodiment of the present invention provides a method for recognizing an object in an image using a camera and a processor. Any one of the objects includes content information and direction information, and the direction information includes a first direction and a second direction. The method includes steps A, B, C, and D, and in step A, if one of the objects is adjacent to the other of the objects in the first direction and no other object is adjacent to the object in the second direction, the processor generates set information corresponding to the object and stores content information of the object in the set information. The method further includes determining the object as a reference object. In step B, if any one of the objects other than the reference object is adjacent to the reference object in the first direction, the processor marks any one of the objects as a follow object and executes step C, and if no object is adjacent to the reference object in the first direction, executes step D. In step C, if the processor determines that the reference object is adjacent to the follow object in the second direction, the processor stores content information of the follow object in the set information and determines the follow object as the reference object and executes step B. In step D, the processor processes the set information to generate a calculation result.

[0005] An embodiment of the present invention provides an augmented reality interaction method for recognizing at least one object in an image using a camera and a processor. The object includes content information and calculation direction information. The calculation direction information includes an inspection direction, a re-inspection direction, or a combination thereof. The interaction method includes steps M, Q, and R. Step M includes providing set information for storing the content information, the calculation direction information, or a result after a processing operation of the processor on at least one object. Step Q includes the processor storing content information of two of the objects in the set information when the objects in the image recognized by the processor are multiple, and in the inspection direction of one of two of the objects, another of the multiple objects is adjacent, and the inspection direction of one of the two of the objects and the re-inspection direction of the other of the two of the objects are opposite to each other. Step R includes the processor transmitting the set information to generate a calculation result.

[0006] An embodiment of the present invention provides an augmented reality interaction method for recognizing at least one object in an image using a camera and a processor. The object includes content information and computation direction information. The computation direction information includes an inspection direction, a re-inspection direction, or a combination thereof. The method includes steps P, Q', and R. Step P includes, if the processor recognizes that the number of the at least one object in the image is 1, the processor stores content information of the at least one object in set information corresponding to the object. Step Q' includes: When the processor recognizes that the number of at least one object in the image is multiple, and two of the multiple objects are adjacent to each other, and the inspection direction of one of the two objects is adjacent to the other of the two objects, and the inspection direction of one of the two objects and the re-inspection direction of the other of the two objects are opposite to each other, the processor stores content information of two of the multiple objects in set information, and step R includes the processor transmitting the set information to generate a calculation result. [Brief description of the drawings]

[0007] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will now be described with reference to the drawings, in which: In accordance with standard practice in the art, many features are not drawn to scale, but are merely illustrative, and dimensions of elements may be arbitrarily expanded or reduced to clearly illustrate features of the present disclosure. [Figure 1] 2 is a flow chart of a method according to an embodiment of the present invention. [Diagram 2] 1 is a flow chart of a method. [Diagram 3] This is a conceptual diagram of an actual usage method. [Figure 4] FIG. 2 is a conceptual diagram of various adjacent objects according to one embodiment of the present invention. [Diagram 5] FIG. 13 is a conceptual diagram of an embodiment of the present invention when an alignment error occurs for some objects. [Figure 6] FIG. 13 is a conceptual diagram of an embodiment of the present invention when an alignment error occurs for some objects. [Figure 7] FIG. 2 is a conceptual diagram of a 3D image when an object is placed in one embodiment of the present invention. [Figure 8] This is a conceptual diagram of interaction between a user and a 3D object. [Figure 9] FIG. 2 is a conceptual diagram of an embodiment of the present invention when arranging some objects. [Figure 10] FIG. 2 is a conceptual diagram of an embodiment of the present invention when arranging some objects. [Figure 11] FIG. 2 is a conceptual diagram of a polygon object according to one embodiment of the present invention. [Figure 12] FIG. 2 is a conceptual diagram of a method according to an embodiment of the present invention. [Figure 13] FIG. 2 is a conceptual diagram of a method according to an embodiment of the present invention. [Figure 14] FIG. 2 is a conceptual diagram of an example of using a method according to an embodiment of the present invention. [Figure 15] FIG. 13 is a conceptual diagram of an example of using a method according to another embodiment of the present invention. [Figure 16] FIG. 13 is a conceptual diagram of an example of using a method according to another embodiment of the present invention. [Figure 17] 13 is a conceptual diagram of equipping a weapon card or an energy card to a character card. [Figure 18] 2 is a flow chart of a method according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, various embodiments or examples will be shown to explain the different features. The examples of specific elements and their arrangements described below are for explaining the present invention. Note that these examples are for illustration only and are not intended to limit the scope of the present invention. Furthermore, although duplicated symbols and notations may be used in different examples, these duplications are intended to simply and clearly describe the present invention and are not intended to represent a specific relationship between the different examples and / or structures described.

[0009] In order to solve the problem of instability when detecting a combination of two or more card patterns at the same time, an embodiment of the present invention provides a method 100 for solving the problem. FIG. 1 is a flowchart of the method 100 of an embodiment of the present invention. The method 100, for example, takes a 2D object photographed by a photographing device as an image and displays it in 3D on a display device. In one embodiment, the method 100 is used in, for example, education and learning. For example, by using an innovative virtual-real interaction mechanism, people's motivation for education and learning can be improved. Furthermore, learning memory can be deepened through the interaction process and visual impression. The application scope may include, but is not limited to, language, words, pinyin, natural science, mathematical formulas, geography, etc. And, the method 100 can be applied to entertainment such as cardboard games, character battle games, character development of public figures, etc., to contribute to brand extension of virtual characters and public figures in games and improve brand value.

[0010] First, in step 101, an object (e.g., a character card) is photographed into a 2D image. Here, the object may be, for example, a card, or other object having image information. In one embodiment, the image may be photographed using various photographing devices. For example, the image may be photographed using a device having a photographing device, such as a mobile phone, a camera, AR glasses, MR glasses, a tablet, a laptop, etc. In more detail, the present invention may be implemented using an iOS / Android device that supports ARKit or ARCore.

[0011] In step 102, the captured 2D image is matched with the created 3D scene. In step 103, during matching, an algorithm may convert the captured 2D image into a coordinate position in the 3D scene. For example, an algorithm may determine the "axis direction" and "distance" of the coordinates, and this information is called aggregate information. Then, in step 104, a first result of the aggregate information is displayed. In step 105, it is determined whether the aggregate information satisfies a combination rule. If the combination rule is satisfied, a second result is displayed in step 106. For example, the corresponding 3D material may be sent to the screen of the mobile device. If not, return to step 104 and judge again. In one embodiment, the aggregate information may be used to store information such as content information, operation direction information, or a result after a processing operation on an object of a processor.

[0012] Hereinafter, the details of step 105 in the method 100 will be described with reference to an embodiment. For example, FIG. 2 is a flow chart of the method 110, showing the details of step 105. FIG. 3 is a conceptual diagram of the actual application method 110. For example, when the "C", "A", and "R" letter cards are arranged in sequence, a 3D image of a vehicle can be generated. In 3(A), the C letter card is arranged, and the content information of each letter card (for example, the inspection direction, re-inspection direction, etc., which will be described later) is recognized using the photographing device and the processor. Next, in FIG. 3(B), the A letter card is arranged, and at this time, the method 110 proceeds to step 111 to perform a first inspection step to generate a first inspection result. In one embodiment, the first inspection step includes using the processor to check whether other letter cards are adjacent to the inspection direction of the C letter card.

[0013] In one embodiment, the inspection direction may be defined as the right side when the character card is correctly placed, but is not limited thereto. In one embodiment, the inspection direction may be the reading direction of an English word, the reading direction of a Chinese word, the placement position of an equipment slot of a character object, or a combination thereof. For example, when there is a relationship between objects, this method can be used. This method may be applied to a board game. For example, when a "character object" indicates the position or direction of the character's back in an object, a "backpack object" may be equipped. A "water bottle object" may be equipped on the backside of the "backpack object". In this example, the rear direction of the character and the backpack is the inspection direction.

[0014] At this time, the distance between the C character card and the A character card is not close enough, so the first check result determined in step 111 is "No", the calculation is stopped, and the calculation result is generated. For example, an image display step is performed to display content information on a display device. For example, an AR screen device (including, for example, a mobile phone, a tablet, AR glasses, etc.) is used to form 3D images of the letters "C" and "A" on the C character card and the A character card, respectively. Next, an R character card is further added to FIG. 3(C), but at this time, the distance between the R character card and the other character cards is also not close enough, so the determination result in step 111 is also "No", and a 3D image of the letter "R" is similarly formed on the R character card. The method 100 according to the embodiment of the present invention can track more than 10 image recognition combinations within the visible range of the camera.

[0015] Next, in FIG. 3(D), the distance between the A and R character cards is reduced. At this time, in step 111, it is determined whether there is an adjacent R character card in the test direction of the A character card, and the first test result is "yes". At this time, the method 110 proceeds to step 112, where the processor checks whether other character cards are adjacent in the retest direction of the A character card, and generates a second test result. In one embodiment, the retest direction may be defined as the left side when the character cards are correctly arranged, but is not limited thereto. At this time, the C character card is not adjacent to the left side of the A character card, so the second test result is "no". If the second test result is "yes", the method returns to step 111 and judges other character cards. For example, in FIG. 3(E), when the judgment is started from the A character, it is determined in step 112 that the C character card is adjacent to the left side of the A character card, so step 111 is executed again for the C character card.

[0016] Note that, here, adjacent does not necessarily mean that the two objects (character cards) are completely attached. In one embodiment, when the first re-inspection step has a tolerance when performing the inspection and there are multiple objects, the adjacent means that the pitch of the inspection direction of the object and the re-inspection direction of the other object in the virtual plane or virtual three-dimensional space is within the tolerance. In one embodiment, the tolerance may include a distance tolerance, and the distance between the two objects may be within the distance tolerance. If the set tolerance is exceeded, the calculation is not continued, so that performance resources can be saved.

[0017] The distance may be calculated based on the center of two adjacent objects, based on the shortest distance between the two objects, or based on a preset distance measurement reference point on the two objects. In one embodiment, the distance tolerance may be 0-5 cm, for example, but is not limited to, the shortest distance between the edge of one of the multiple objects and the other of the multiple objects may be less than 5 cm. In one embodiment, other distance tolerances (for example, less than 10 cm) may be included as necessary. For example, when the content information of one of the two objects is a magic ball or a talisman in a card game, the talisman or magic ball is "moved" or "oriented" a certain distance toward the other object after being displayed on the display device. In one embodiment, referring to FIG. 16, for example, when the energy card 303 and the character card 301 in FIG. 16(A) and (B) are combined, the 3D object of the energy card moves directional toward the character card. 17, when the equipment card 304 or the energy card 303 is within the distance tolerance range, the 3D object moves directionally towards the character card 301. In one embodiment, when the distance tolerance is within different ranges, a corresponding result, such as a video, is generated. For example, when the distance tolerance of two objects is 0-5 cm, 5-10 cm, and 10-15 cm, the processor generates different calculation results, respectively, and can achieve various results.

[0018] In one embodiment, the tolerance may include an angle tolerance, and in this case, the inspection direction and the re-inspection direction satisfying the pairing relationship means that the included angle between the two is within the angle tolerance. For example, the angle tolerance may be 0 degrees to 15 degrees, or 80 degrees to 110 degrees, but is not limited thereto. In another embodiment, any angle may be set in advance to achieve the object of the method of the present invention. For example, a relatively wide tolerance may be set so that the pairing relationship is established between the two objects in all directions. This provides the user with a relatively large margin of error. For example, by setting multiple distance values, multiple combination relationships are established between the two objects. For example, when the distance between the two objects is 5 to 10 cm, a first set of results or videos can be generated. When the distance between the two objects is 0 to 5 cm, another set of different results or videos can be generated. With such settings, the degree of freedom is increased by the distance between the objects in addition to the objects themselves, and the versatility of the method can be enhanced.

[0019] In one embodiment, the angle is determined according to the following formula: JPEG0007606061000001.jpg1276 The distance is determined by the following formula: That is, if the relationship between the coordinates of the two objects is known, the distance and angle between the two cards can be determined using the above equations (1) and (2).

[0020] For example, FIG. 4 is a conceptual diagram of various adjacent objects in one embodiment of the present invention. The C, A, and R letter cards in FIG. 4(A) are arranged in the X direction and have approximately the same Y coordinate. The C, A, and R letter cards in FIG. 4(B) to FIG. 4(E) are also arranged in the X direction, but do not have exactly the same Y coordinate. In this case, the inspection direction may be the +X direction and the re-inspection direction may be the -X direction, but is not limited to this. That is, in the Y direction, there may be a tolerance between each letter card so that a desired image is displayed even if the user does not perfectly align each letter card.

[0021] In addition, when the first check result is "yes" and the second check result is "no", if one of the at least one object is adjacent to the other of the at least one object in the check direction and the other of the at least one object is not adjacent to the one of the at least one object in the re-examination direction, the A letter card represents that it is the first of the aligned letter cards, so at this time, the A letter card may be defined as the "leading object". In this way, when there is a leading object, comparison (hierarchical relationship) of other objects can be performed. For example, when rearranging English letters, since there are no other objects in the re-examination direction of the leading object, the beginning of the sentence becomes the leading object. However, in one embodiment, even if the above conditions are not satisfied, it can be the leading object. For example, the leading object may be content information previously set as a character object, a construction base object, or other changeable object recognized by the processor. In addition, depending on the situation, there may be multiple leading objects so as to have multiple sets of adjacent cards at the same time. In one embodiment, the leading object may be called a sentence beginning object, a base object, a character object, or the like. The base object may have different facility objects constructed around it, and the character object may have different equipment objects mounted on it.

[0022] If the first check result in step 111 is "no", that is, if it is determined that there is no other character card to the right of the character card at the first time, the subsequent check (for example, checking whether there is another character card to the left) is not performed, thereby saving the processor's operating time and improving the calculation efficiency. Also, in step 111, it is continuously checked whether other objects (character cards) satisfy the requirements of step 111.

[0023] Next, the method 110 proceeds to step 113. The processor performs a first re-examination step according to the arrangement order of the letter cards. Referring to FIG. 3(D), at this time, the processor checks whether the examination direction of the A letter card and the re-examination direction of the R letter card satisfy the pairing relationship. (In system execution, the processor asks the card on the right side of the A letter card (i.e., R) which card is on the left side of the R letter card. If it is confirmed that the left side of the R letter card is also an A letter card, it is confirmed whether the pairing relationship is formed.) For example, an arrangement such as "CA" and "AR" may be defined as satisfying the pairing relationship. Although other arrangement methods (e.g., "RC", "RA", etc.) also satisfy the pairing relationship, since such arrangements are not stored in the database, no data is generated as a result of the comparison, and only the independent images of R and C letters are displayed. At this time, since the A letter card and the R letter card satisfy the pairing relationship, the processor processes the content information of the A letter card and the R letter card and generates a calculation result such as pairing success or pairing failure. The A letter card and the R letter card in FIG. 3(D) are an example of successful pairing, and an example of unsuccessful pairing is when the letter cards are arranged diagonally or the axial positions are different, as shown in FIG. 6(A). The A letter card is set as the reference object, and the R letter card is set as the follow object. If the pairing result does not satisfy the pairing relationship, the processor stops the execution of the first re-examination step. In one embodiment, the arrangement order may include an order in which at least one object is arranged and aligned according to the inspection direction to which it belongs, an order in which at least one object is arranged and aligned according to the inspection direction to which the leading object belongs, or a combination thereof.

[0024] In one embodiment, the follow object may be adjacent to the reference object in its retest direction. In one embodiment, the lead object may be a starting object that starts performing the first retest step or the first test step, and the test direction of the lead object may be a direction in which the processor performs the first retest step or the first test step on at least one object.

[0025] In one embodiment, the processor stores the pairing relationship in the set information, and generates a calculation result based on the set information. In one embodiment, the processor does not perform storage, so that the operation efficiency can be improved. In one embodiment, if the pairing relationship is satisfied, the processor may update the content information (which may be a null value) of the object. In one embodiment, the content information, the inspection direction, and the re-inspection direction may exist independently. That is, in some situations, the object may not have content information, but may have multiple calculation direction information. For example, the object may have only multiple calculation direction information, such as a construction base, a character card, etc. In one embodiment, the multiple calculation directions may include an inspection area (for example, a virtual plane, a virtual three-dimensional space, or a combination thereof in the recognition image of the processor, or an X-axis, a Y-axis, a Z-axis, or a combination thereof in the virtual three-dimensional space of the object), and if any object is included in the inspection area of ​​the object, a first re-inspection step is performed.

[0026] Next, in step 114, the processor sets the reference object (A letter card) as the judged object and sets the follow object (R letter card) as the reference object. After that, in step 115, a second check step is performed to determine whether there is another letter card behind the current reference object (R letter card). That is, it is determined whether there is another letter card in the check direction of the reference object (R letter card). In the state of FIG. 3(D), since there is no other letter card behind the R letter card, the process skips step 115 and proceeds to step 116, the operation of method 110 is terminated, and the result after the calculation is displayed. In one embodiment, if the R letter card is the last card, it is not the reference object. The reason is that if the processor performs the first check step on any object and the result is "no", the calculation is directly stopped and the first check result of the R letter card becomes "no". Therefore, it is not necessary to perform step 113 again, so that the performance resources required during the calculation can be saved.

[0027] However, at this time, only the A letter card and the R letter card are combined, and do not form a complete word, so the result after the calculation at this time still displays the 3D images of the A letter card and the R letter card.

[0028] Next, in FIG. 3(E), the C character card is brought closer. As in the logic of the judgment, after passing through step 111, at the end of step 112, the C character card is judged as the leading object, and in step 113, the A character card is judged as the following object. Next, in step 114, the C character card is set as the determined object, the A character card is set as the new reference object, and the R character card is set as the new following object. At this time, since the R character card is still behind the A character card, the calculation result in step 115 of the method 110 is "pass" (there is another object behind the reference object), and the method returns to step 113 to judge the subsequent A and R character cards until the complete character string is judged. In one embodiment, when there are multiple object arrays, the processor may repeatedly execute steps 113, 114, and 115 until the last object is judged (the R character card in FIG. 3(E)). At this time, the calculation can be stopped, and there is no need to repeatedly execute the steps, so that the performance resources required during the calculation can be saved.

[0029] Next, referring to FIG. 3(F) to FIG. 3(I), the processor at this time judges that the letters are arranged in the order of C, A, and R, so that the original 3D images of "C", "A", and "R" shrink and disappear (FIG. 3(F)) based on the judgment result at this time (the second result of method 100), and a 3D image of a vehicle (FIG. 3(G) to FIG. 3(I)) is displayed to inform the user that the arrangement is correct. After that, in FIG. 3(J), the R letter card is moved away, and method 110 judges that "C" and "A" satisfy the arrangement method, but the processor cannot recognize the word in the arrangement, and therefore cannot compare it with the contents of the database, and therefore the corresponding vehicle image is not displayed. However, the present invention is not limited to this. For example, even if the user removes one of the recognition images (for example, one of the C, A, and R letter cards), the processor can recognize the content information in the image and display it on the display device. For example, if the R card is removed, the 3D image of the vehicle is not displayed above the C, A, and R cards, but the 3D images of the three letters C, A, and R are displayed. At this time, the user can simultaneously check the information of the object itself and the image corresponding to the object. This allows the user to learn using the auxiliary images.

[0030] In the method 110, since only one object (character card) is determined in one step, the data of each object is calculated separately, which can save performance resources and avoid affecting the display function.

[0031] In one embodiment, the content information may include model information, three-dimensional model information, character model information, audio information, digital media information, character information, content recognition information, authentication information, virtual object information, encryption information, interaction command information, or a combination thereof. In one embodiment, the content information may also be a null value. In one embodiment, the content information includes an interaction object for a user and an object to interact with each other. For example, the interaction object may be printed on the surface of the object. For example, by printing an interaction object on a character figure of a board game, the interaction object can be a part that is precisely pressed or interacted with by a finger during operation, and can also be used to block a specific area or realize the effect of image recognition. For example, an interaction method can be realized by an image design on an object, and a specific area can be set on a character object, and a processor can determine a new inspection image according to a specific area that a user blocks the image, thereby realizing a new combination of game play.

[0032] In one embodiment, the processor may include a general purpose processor, a chip multiprocessor (CMP), a dedicated processor, an embedded processor, a digital signal processor (DSP), a network processor, an input / output (I / O) processor, a media access control (MAC) processor, a radio baseband processor, a co-processor, a complex instruction set computer (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, and / or a very long instruction word (VLIW) microprocessor or other processing device microprocessor, etc. The processor may also include a controller, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a programmable logic device (PLD), etc.

[0033] 5A to 5C are conceptual diagrams of some object arrangement errors in an embodiment of the present invention. As shown in Fig. 5(A) to Fig. 5(C), the C, A, and R letter cards are arranged in the X direction, but the arrangement order is not CAR. That is, the hierarchical order does not match the corresponding database data. In such a situation, the determination result in step 114 is "No." Therefore, as shown in Fig. 5(D) to Fig. 5(F), even if the three letter cards are brought close to each other, only the 3D letter images of C, A, and R are displayed, and no vehicle is displayed.

[0034] FIG. 6 is a conceptual diagram of some object arrangement errors in an embodiment of the present invention. In such a situation, the direction of the character card does not completely satisfy the above rule. For example, the inspection direction is defined as the +X direction of the character card, and the re-inspection direction is defined as the -X direction of the character card. As shown in FIG. 6(A), the A character card is found in the inspection direction of the C character card, but the C character card is not found in the re-inspection direction of the A character card. Instead, the R character card is found in the re-inspection direction (-X direction) of the A character card, that is, the two objects do not satisfy the pairing relationship. At this time, the result of step 113 is "No". Also, as shown in FIG. 6(B), there is no character card in the inspection direction of the C character card. There is a C character card in the inspection direction of the A character card, but there is no character card in the re-inspection direction of the C character card, so the pairing relationship is not satisfied. At this time, the result of step 111 is also "No". Therefore, as shown in FIG. 6(C) to FIG. 6(E), even if the three letter cards are brought close to each other, only the three 3D letter images of C, A, and R are displayed, and the vehicle is not displayed.

[0035] In one embodiment, the 3D character image may be rotated when displayed, allowing the user to determine whether the placement position is correct. For example, the following are conceptual diagrams of a 3D image when an object is placed in one embodiment of the present invention. As shown in Figs. 7(A) to 7(E), the placement position is correct, so the correct "R" character can be confirmed at the start and end points of the animated display. Also, as shown in Figs. 7(F) to 7(J) and Figs. 7(K) to 7(O), the placement direction of the R character card in these two situations is not accurately facing the user, so the correct "R" character cannot be confirmed at the start and end points of the animated display. This is advantageous for the user to determine the correct placement direction.

[0036] In one embodiment, a user can interact with a 3D object by gestures. The gestures may include covering, rotating, clicking, double-clicking, multi-touching, moving, and tapping the object. When a 3D object interaction is displayed on the screen, the gestures include operations such as clicking, double-clicking, multi-touching (including rotating), and moving. During the interaction, the processor may update the content information of the object. For example, a conceptual diagram of an interaction between a user and a 3D object is shown in FIG. 8(A). A vehicle door can be opened by a gesture. Note that a tablet computer is used as the display device here, but the present invention is not limited thereto. For example, other iOS / Android display devices supporting ARKit or ARCore, such as mobile phones, AR glasses, laptops, screens, and televisions, may be used. In FIG. 8(B), the vehicle lights can be turned on. In FIG. 8(C) and FIG. 8(D), the color of the vehicle can be changed. This allows the user to operate the object.

[0037] In the above embodiment, the C, A, and R letter cards are arranged in order to form an image of a vehicle, but the present invention is not limited to this. For example, Fig. 9 is a conceptual diagram of an arrangement of some objects in an embodiment of the present invention. As shown in Fig. 9(A) to Fig. 9(F), the A, R, and C letter cards are arranged in order to generate the word "ARC" (arc), and a 3D video of the arc can be displayed.

[0038] In one embodiment, additional cards may be added. For example, FIG. 10 is a conceptual diagram of some object arrangements in an embodiment of the present invention. As shown in FIG. 10(A) to FIG. 10(D), an additional D letter card may be added to form the word "CD" with the C letter card. As shown in FIG. 10(E) to FIG. 10(J), an additional V, N letter card may be added to form the word "VAN" with the A letter card. This provides the user with more combination options.

[0039] In the above embodiment, a rectangular card having characters is illustrated, but the present invention is not limited thereto. For example, the present invention may be applied to any polygonal object. FIG. 11 is a conceptual diagram of a polygonal object 200 according to an embodiment of the present invention, in which a pentagon is taken as an example. In an embodiment, the inspection direction of the polygonal object 200 may include an inspection vector, and the re-inspection direction may include a pre-set re-inspection vector. The inspection vector and the re-inspection vector may include information on "direction" and "size" (vector length) to be used as the basis for the pairing relationship, inspection step, or re-inspection step. In addition, virtual coordinates may be assigned to a plurality of objects, and a relative positional relationship may be established between the plurality of objects. As a basis for determining the pairing relationship, inspection step, or re-inspection step, for example, the relative positional relationship between the objects may be determined by calculating the distance based on the difference value of the coordinate information of the two objects.

[0040] As shown in Fig. 11, the perpendicular direction of each side of the object may be the inspection direction, the re-inspection direction, or a combination thereof. For example, a pentagon may include five sides 201, 202, 203, 204, 205 and five vectors 211, 212, 213, 214, 215 perpendicular to each side 201, 202, 203, 204, 205, respectively. More specifically, if the vectors 211, 212, 213, 214, 215 represent the inspection direction and the re-inspection direction, axial information is preset for a plurality of objects, and the inspection direction is set by the vectors 211, 212, 213, 214, 215. In this way, when a plurality of polygonal objects 200 are arranged with respect to each other, it is determined whether the arrangement satisfies the preset information by the method 100 and the method 110.

[0041] In one embodiment, the pairing relationship may refer to a preset inspection threshold. For example, it may refer to an error tolerance that can be recognized by a processor. For example, if the difference value between the inspection vector and the re-inspection vector is zero, that is, the inspection direction of an object and the re-inspection direction of another object adjacent to it are opposite, they are "opposite directions" to each other. That is, they have the same magnitude and opposite direction. However, since a user of the method of the present invention (e.g., a child) may place each object diagonally, the difference value may not be exactly zero. In addition, the inspection threshold (or tolerance) preset by the system enhances the user experience of the method of the present invention. For example, when the objects are placed in a position close enough (e.g., in the situation of Figure 4 (B) to Figure 4 (E)), it can be determined that the character cards are adjacent to each other. The vectors may be defined in multiple axial directions on the object directly. These vectors may be defined as "axes".

[0042] In one embodiment, each object may have its own inspection threshold for content information, which may be different from each other. For example, by setting a relatively wide inspection threshold in a specific inspection direction of a specific object, it can be more easily "equipped, mounted, and placed" than with other objects, and the pairing relationship can be easily inspected or re-inspected by the processor. For example, the "construction base" may be a physically relatively large object or card. Also, by providing various different inspection thresholds, different results can be obtained in different relative positional relationships. For example, when two objects are in close contact or attached to each other, the processor can determine one of the multiple inspection thresholds, and when the two objects are slightly apart, the processor can determine another of the multiple inspection thresholds. In one embodiment, the inspection threshold may include a mathematical operation result of any two pitches of the objects (e.g., letter cards), a mathematical operation result of any two vectors, a calculation result of any two content information, or a combination thereof.

[0043] Another embodiment of the present invention further provides a method 120 that can be used to execute the details of step 105, for example as an interaction method for augmented reality. FIG. 12 is a conceptual diagram of the method 120 of an embodiment of the present invention. As shown in FIG. 12, in step 121, it is determined whether the arrangement of each object satisfies a first rule. In one embodiment, the first rule may include determining whether another object is adjacent to one of the objects in a first direction and no object is adjacent to the object in a second direction. For example, as shown in FIG. 3(D), the first direction may be the right side of the character card and the second direction may be the left side of the character card. At this time, the R character card is adjacent to the A character card in the first direction, and no other character card is adjacent to the A character card in the second direction. At this time, the result of the determination is "yes," and the method 120 proceeds to step 122, where the processor generates set information corresponding to the object, stores content information of the object in the set information, and defines the object as a reference object. That is, in this case, the A character card is defined as a reference object.

[0044] In one embodiment, the content information includes relative position information, for example, multiple direction information of multiple objects relative to a reference object. In one embodiment, the content information may include character information, content recognition information, authentication information, virtual object information, encryption information, interaction command information, or a combination thereof. In one embodiment, the set information may include content information of the object, and may include at least one character information, multiple character origin information in order, combination information of pre-set characters and equipment, or a combination thereof. Note that the content information of multiple objects may be the same. For example, the content information of two objects may both be the letter "B".

[0045] In addition, in step 121, if the determined object does not satisfy the first rule (for example, if there is no other object adjacent to the right side of the C character card in FIG. 3(D)), step 121 is executed for the other objects until a reference object is found.

[0046] Next, in step 123, it is determined whether the arrangement of each object satisfies the second rule. In one embodiment, the second rule may include determining whether an object other than the reference object is adjacent to the reference object in a first direction. If the determination result is "yes", the method 120 proceeds to step 124, where the processor displays any object other than the reference object as a follow object. For example, in the example of FIG. 3(D), if it is determined that the R character card is adjacent to the A character card in the first direction, the R character card is displayed as a follow object. Next, in step 125, it is determined whether the arrangement of each object satisfies the third rule. In one embodiment, the third rule may include the processor determining whether the reference object is adjacent to the follow object in a second direction. For example, it is determined whether another character card is adjacent to the R character card in the second direction in FIG. 3(D). In FIG. 3(D), the left side of the R character card is the A character card, so the determination result at this time is "yes". In one embodiment, if the result of executing step 121 is “yes”, for example, if the R letter card is adjacent to the A letter card in the first direction, the first rule is satisfied, and in this case, step 122 is executed, A and R are stored in the set information, and step 124 is executed to make the R letter card the follow object without the need to execute step 123 again, which is to “determine whether an object other than the reference object is adjacent in the first direction”.

[0047] If the determination result of step 125 is "yes", the method 120 proceeds to step 126, where the processor stores the content information of the following object in the collection information, determines the following object as the reference object, and returns to step 123. That is, the reference object at this time is the R character card. Next, execute step 123. Since no other object is adjacent to the reference object (the R character card) in the first direction at this time, the determination result of step 123 is "no". If the determination result is "no", that is, if no object is adjacent to the reference object in the first direction, the method 120 proceeds to step 127, where the processor processes the collection information to generate a calculation result. The generated calculation result may include a comparison with a content database by the processor. If the collection information corresponds to the content in the database, the calculation result is displayed on the display device. For example, in FIG. 3(D), a 3D image of the A and R character cards is displayed on an AR screen device (including mobile phones, tablets, AR glasses, etc.). In one embodiment, the display device may display a prompt image, a word image, a success prompt image, an error prompt image, a failure prompt image, a character animation image, a character skill image, or a combination thereof based on the calculation result.

[0048] Similarly, when the arrangement of the C, A, R letter cards is completed, the method 120 may be executed with the same logic as shown in Figures 3(E) to 3(I), and the 3D image of the vehicle may be displayed in the final step 127. The above calculation results may correspond to signals for displaying images, sounds, images, output results, or the like on a display device.

[0049] Also, if the result of the determination in step 125 is "no", the method 120 may proceed to step 127 to display an image corresponding to the calculation result on the display device. In one embodiment, if the result of the determination in any step in the method 120 is "no", the processor may store content information of the reference object and other objects other than the reference object in an error record set. For example, if the word CARRY is misspelled as CERRY, the processor may store content information of the C object and the E object in an error record set so that the user can check the error part. The method 120 judges only one object at a time, and if the rule is not satisfied, the judgment is terminated and no other subsequent calculations are performed, thereby saving performance resources.

[0050] The embodiment of the present invention further provides a method 130 that can be used to perform the details of step 105, for example, as an interaction method of augmented reality. FIG. 13 is a conceptual diagram of the method 130 of an embodiment of the present invention. As shown in FIG. 13, in step 131, content information, multiple operation direction information (e.g. multiple inspection directions), or object set information is provided for storing the result after the processor performs processing operation on at least one object. Then, in step 132, the number of objects is determined. If the number of objects is odd, the process proceeds to step 133, where the processor stores the content information of the object in the set information corresponding to the object, and in step 134, the processor transmits the set information to generate the operation result. For example, in FIG. 3A, if there is only one C character card, the operation result in step 134 displays a 3D image of the C character card.

[0051] If the result of the determination in step 132 is multiple objects, the method 130 proceeds to step 135, where if it is determined that two of the objects are adjacent to each other, and one of the objects is adjacent to the other object in the inspection direction, and the other object is not adjacent to any other object in the re-examination direction, the processor marks the object as a leading object. For example, in FIG. 3(D), the A character card is marked as a leading object, or in FIG. 3(E), the C character card is marked as a leading object. In one embodiment, the leading object may be a sentence head object, a character object, a construction base object, or any other person, item, character, or event that can be added, added, or appended.

[0052] Next, in step 136, if the number of objects in the image recognized by the processor is more than one, and one of the objects is adjacent to the other object in the inspection direction, and the inspection direction of the object and the re-inspection direction of the other object are opposite to each other, the processor stores the content information of the two objects in the set information. In one embodiment, step 136 may further include a condition that the two objects are adjacent to each other. This makes it possible to confirm that the two objects are adjacent and correctly arranged, for example, as in the A and R letter cards in FIG. 3(D).

[0053] Next, in step 137, the processor marks a reference object, such as the A character card in Fig. 3(D) or the C character card in Fig. 3(E). If another object is adjacent to the reference object in the inspection direction, the processor marks the other object as a follow object. After that, if the inspection direction of the reference object and the re-inspection direction of the follow object are opposite to each other, the processor checks the content information of the reference object and the content information of the follow object in the set information.

[0054] The method 130 proceeds to step 138, where the processor marks the reference object as the inspected object, and then the processor marks the follow object as the reference object, and marks other objects adjacent to the inspection direction of the reference object as other follow objects. For example, in FIG. 3(E), originally the C character card is the reference object, and the A character card is the follow object. After inspecting the C character card, the A character card is re-marked as the reference object, and the R character card is marked as another follow object. Next, in step 139, it is determined whether the inspection direction of the new reference object and the re-inspection direction of the new follow object are opposite to each other. That is, it is determined whether there are other following follow objects. If yes, the processor stores the content information of the new reference object and the content information of the new follow object in the set information. If not, the processor executes step 134 to send the set information by the processor to generate the calculation result. The above method 130 judges only one object at a time, and if the rule is not satisfied, the judgment is terminated and other subsequent calculations are not performed, so that performance resources can be saved.

[0055] In the above embodiment, the arrangement of character cards has been described as an example, but the present invention is not limited thereto. The method of the present invention can also be applied to board games, for example. For example, a conceptual diagram of an example using a method according to an embodiment of the present invention is shown in FIG.

[0056] First, in Fig. 14(A), character card 301 is placed, and a female warrior equipped with dual swords appears. Next, in Fig. 14(B), when the position of character card 301 is clicked, button 305 appears. Next, when attack button 305 is clicked in Fig. 14(C), the character begins an attack in Fig. 14(D). In one embodiment, if attack button 305 does not appear, it may be designed so that the subsequent equipment card 302 or energy card 303 cannot be equipped.

[0057] Next, in FIG. 14(E)-FIG. 14(F), an additional katana weapon card 302 is moved closer to the character card 301 until the distance between the character card 301 and the weapon card 302 is smaller than the inspection threshold or tolerance. This is a conceptual diagram of an example using a method according to another embodiment of the present invention. Next, in FIG. 15(A)-FIG. 15(B), the weapon 3D image on the weapon card 302 moves onto the character card 301 to complete the combination. In FIG. 15(B), when the katana 3D image on the katana weapon card 302 is equipped to the 3D image on the character card 301, the character attacks with the katana instead of the dual swords. After that, in FIG. 15(C)-FIG. 15(D), when the character card 301 is combined with the weapon card 302 for the first time, the system automatically shows the attack animation in FIG. 15(C)-FIG. 15(D) without the need to click the attack button. In Figures 15(G) and 15(H), when the attack button 305 is clicked again, the character attacks again with the military sword. In this situation, the character card 301 may be, for example, a leading object in the above method, and the weapon card 302 may be a following object in the above method. Next, in Figures 15(E) and 15(F), since the content information of each object can be updated in the above method, the weapon of the character card 301 at this time is updated to a military sword, and even if the weapon card 302 is removed, the weapon does not return to the dual swords.

[0058] FIG. 16 is a conceptual diagram of an example using a method according to another embodiment of the present invention. As shown in FIG. 16(A), when the energy card 303 is brought close to the character card 301, the 3D image on the energy card 303 may move to the character card 301 as shown in FIG. 16(B) to FIG. 16(D). For example, the energy card 303 may move in a direction 306 toward the character card 301. Next, as shown in FIG. 16(E), the attack method of the character may change accordingly. In the embodiment of FIG. 15, after the weapon card 302 is removed, the attack method of the character card 301 does not change even if the weapon card 302 is removed. That is, the content information is not updated again even if the weapon card 302 is removed, but the present invention is not limited to this. For example, as shown in FIG. 16(F) and FIG. 16(G), after the energy card 303 is removed, the character's attack method returns to the previous attack method. This can provide the designer with greater design flexibility. In one embodiment, the above-mentioned weapon cards 302 and energy cards 303 may be directly equipped to the character card 301 without any particular order. In one embodiment, the energy card 303 cannot be added or used unless the weapon card 302 is equipped to the character card 301. That is, each object may have a particular combination order.

[0059] In one embodiment, if the 3D image on the weapon card 302 or the energy card 303 has not moved onto the character card 301, the movement of the 3D image may be stopped until the weapon card 302 or the energy card 303 moves into the detection range again. In one embodiment, a command may be provided to restore the content information of each card to its initial content information. For example, the weapon card 302 or the energy card 303 may be collected by controlling with a gesture. In one embodiment, multiple weapon cards 302 or energy cards 303 may be equipped at the same time.

[0060] 17 is a conceptual diagram of equipping a weapon card 302 or an energy card 303 to a character card 301. When at least a part of the weapon card 302 or the energy card 303 comes within the detection range 304 of the character card 301, it is determined that the distance between the character card 301 and the weapon card 302 (or the energy card 303) is smaller than the threshold or allowable value of the inspection, and subsequent determinations may be continued.

[0061] FIG. 18 is a flow chart of a method 400 according to an embodiment of the present invention. The method 400 is for determining the combination relationship of the character card 301, the weapon card 302, and the energy card 303 according to the embodiment. First, in step 401, an object (e.g., the character card 301, the weapon card 302, and the energy card 303) is photographed into a 2D image. The object here may be, for example, a card or an object having other image information. In one embodiment, the image may be photographed with various photographing devices, such as a mobile phone, a camera, an AR glass, an MR glass, a tablet, a laptop, or other devices equipped with an IOS / Android photographing device supporting ARKit or ARCore.

[0062] In step 402, the captured 2D image is aligned with the generated 3D scene. In step 403, during alignment, an algorithm may convert the captured 2D image into a coordinate position in the 3D scene. For example, the algorithm may determine the "axis direction" and "distance" of the coordinates, and the information may be aggregated.

[0063] In step 404, the processor may determine whether the user has turned on an attack command. If the user has turned on an attack command, in step 405, it is determined whether the user has executed the attack command. If the user has not turned on an attack command or has not executed an attack command, step 404 or step 405 proceeds to step 406, and at this time, even if the weapon card 302 is provided, the character card 301 and the weapon card 302 are not combined.

[0064] If it is determined in step 405 that the user has executed an attack command, then in step 407 the data on the character card 301 is updated, for example, the character performs an attack. Next, in step 408, the weapon card 302 is added. In step 409, it is determined whether the distance between the weapon card 302 and the character card 301 is within the detection range 304. If the answer is "yes," the process proceeds to step 410, where the data on the character card 301 is updated. For example, the character is equipped with a new weapon. If the answer is "no" in step 409, the process returns to step 406, and no combination of the character card 301 and the weapon card 302 is generated.

[0065] Next, in step 411, the energy card 303 is added. Then, in step 412, it is determined whether the distance between the character card 301 and the energy card 303 falls within the detection range 304. If yes, the data of the character card 301 is updated, for example, the attack method of the character card 301 is changed, and step 412 is repeatedly executed until the distance between the character card 301 and the energy card 303 falls outside the detection range 304. At this time, the method 400 proceeds to step 414, where the combination of the character card 301 and the energy card 303 does not occur.

[0066] As described above, the embodiment of the present invention provides a method for recognizing combinations of objects in various augmented reality images. The camera and the processor can be used to recognize objects in the images. This saves the performance resources required during processing, solves the problem of instability when checking combinations of two or more card patterns at the same time, and provides functions such as entertainment and education. In addition, the interaction method provided by the embodiment of the present invention can increase the flexibility of the combination of each object.

[0067] Although specific embodiments and advantages of the present invention have been disclosed, it should be understood that those of ordinary skill in the relevant art may make changes, substitutions, and modifications without departing from the spirit and scope of the present invention. In addition, the scope of protection of the present invention is not limited to the processes, machines, manufacture, material compositions, devices, methods, and procedures described in the specific embodiments herein, which are known to anyone of ordinary skill in the art. It is understood that processes, machines, manufacture, material compositions, devices, methods, and programs currently or developed in the future can be used in accordance with the present disclosure as long as they perform substantially the same functions, or as long as they are developed in the future. The embodiments described herein achieve substantially the same results. Therefore, the scope of protection of the present invention includes the above-mentioned processes, machines, manufacture, material compositions, devices, methods, and programs. In addition, each patent application scope constitutes a separate embodiment, and the combination of each patent application scope and the embodiment is also included in the scope of protection of the present invention. [Explanation of symbols]

[0068] 100,110,120,130,400: Method 101,102,103,104,105,106,111,112,113,114,115,116,117,118,121,122,123,124,125,126,131,132,133,134,135,136,137,138,139,401,402,403,404,405,406,407,408,409,410,411,412,413:Step 200: Polygon object 201, 202, 203, 204, 205: Side 211, 212, 213, 214, 215: Vector 301: Character card 302: Weapon Card 303: Energy Card 304: Detection range

Claims

1. 1. An augmented reality interaction method for recognizing at least one object in an image using a camera and a processor, comprising: The at least one object includes content information and a plurality of pieces of calculation direction information; the plurality of pieces of calculated direction information include an inspection direction, a reinspection direction, or a combination thereof; The method comprises: A first inspection step; a first re-examination step including the processor checking whether the inspection direction of one of the at least one object and the re-examination direction of any one of the at least one object adjacent to the one object satisfy a pairing relationship; If the pairing relationship is satisfied, processing the content information of the one of the at least one object and the any one of the at least one object by the processor to generate a calculation result; determining, by the processor, one of the at least one objects as a leading object when the one of the at least one objects is adjacent to the one of the at least one objects in the inspection direction and the one of the at least one objects is not adjacent to the other of the at least one object in the re-inspection direction; The first inspection step includes: The processor checks whether any one of the at least one object is adjacent in the check direction of one of the at least one object to generate a first check result; if the first test result is "yes", testing by the processor whether one of the at least one objects is adjacent to any other of the at least one object in the re-examination direction to generate a second test result; The method further includes performing, by the processor, the first re-examination step starting from the top object; The first re-inspection step further includes: The processor checks whether the pairing relationship is satisfied and then generates a pairing result; and the processor processes the pairing result to generate the calculation result.

2. 2. The method of claim 1, wherein if the first check result of the first check step is "no", the processor repeats the first check step to check another of the at least one object.

3. The processor checks whether a re-examination direction of a follow object and the inspection direction of the at least one object satisfy the pairing relationship to generate a pairing result, and the follow object is adjacent to the at least one object in the re-examination direction to which the follow object belongs, The method of claim 1 , wherein the processor processes the pairing result to generate a calculation result.

4. 2. The method of claim 1, wherein if the processor determines that the first test result of one of the at least one object is "yes" and the second test result is "no", the processor processes one of the at least one object as the leading object.

5. The first re-inspection step further includes: said processor performing said first re-examination step on said at least one object according to its arrangement order; The method of interaction according to claim 1 or 3, further comprising: if the pairing result does not satisfy the pairing relationship, the processor stops the first recheck step.

6. The first re-inspection step further includes: said processor performing said first re-examination step on said at least one object according to its arrangement order; If the pairing result does not satisfy the pairing relationship, the processor stops the first rechecking step; The interaction method according to claim 1, characterized in that the arrangement order is selected from an order in which the at least one object is arranged and aligned according to the inspection direction to which the at least one object belongs, an order in which the at least one object is arranged and aligned according to the inspection direction to which a leading object belongs, or a combination thereof.

7. The interaction method according to claim 1 , wherein the number of the leading objects is more than one.

8. The interaction method of claim 1, wherein the leading object includes the pre-defined content information as a character object, a building base object, a combination thereof, or other additional, appendable, or addable person, item, character, or event recognized by the processor.

9. the number of the at least one object is a plurality; the inspection direction includes a preset inspection vector; the retest direction includes a preset retest vector; The method of claim 1 , wherein the pairing relationship includes a pre-established pairing relationship.

10. the plurality of objects includes a first object and a second object, the first object includes a first object inspection threshold; the second object includes a second object inspection threshold; the first object inspection threshold is different from the second object inspection threshold; The pairing relationship includes a first pairing check threshold and a second pairing check threshold; The augmented reality interaction method further includes: Checking whether a mathematical operation result of the distance between the first object and the second object, a mathematical operation result of the vector of the first object and the second object, a calculation result of the content information of the first object and the second object, or a combination thereof satisfies the first pairing check threshold or a second pairing check threshold; The interaction method according to claim 9 , further comprising displaying the calculation result based on the first pairing check threshold or the second pairing check threshold.

11. Furthermore, storing the pairing relationships in at least one collection of information by the processor; The method of claim 1 , further comprising: the processor processing the at least one set of information to generate a computation result.

12. The first re-inspection step further includes: If an inspection direction of one of the at least one objects and the re-inspection direction of the one of the at least one objects satisfy the pairing relationship, the processor sets the one of the at least one objects as a reference object, and the processor sets the one of the at least one objects as a follow object; If the inspection direction of the reference object and the re-inspection direction of the follow object satisfy the pairing relationship, the processor processes the content information of the reference object and the follow object to generate a calculation result; the processor setting the reference object as a determined object and the follow object as the other of the reference objects; After the first re-inspection step, the processor executes a second inspection step, the second inspection step including setting the at least one object as the other of the follow objects if the at least one object is adjacent in an inspection direction of the other of the reference objects; 2. The method of claim 1, wherein the processor repeatedly executes the first recheck step and the second check step until the result of the second check step is "no".

13. The method of claim 1 , wherein if the pairing relationship is satisfied, the processor updates the content information of one of the at least one object.

14. The interaction method according to claim 13, characterized in that, when the user removes the one of the at least one object in the recognized image, the processor reads the content information of the one of the at least one object and displays it on a display device.

15. Further, the image display step is included, the processor recognizing the content information of at least one object in an image and displaying the content information on a display device; 2. The interaction method of claim 1, wherein the content information includes a null value or includes model information, three-dimensional model information, character model information, voice information, digital media information, text information, content recognition information, authentication information, virtual object information, encryption information, interaction command information, or a combination thereof.

16. the content information includes an interaction object for allowing a user to interact with the at least one object; the interaction object is printed on a surface of the at least one object; the user interacts with the at least one object through a gesture; The method of claim 1 , wherein the processor updates the content information of the at least one object.

17. The plurality of pieces of calculated direction information include an inspection area, The method of claim 1 , further comprising: performing the first re-inspection step if any one of the at least one object is included within the inspection area of ​​one of the at least one object.

18. 18. The interaction method according to claim 17, wherein the inspection area includes a virtual plane, a virtual three-dimensional space, or a combination thereof in an image recognized by a processor, and the plurality of pieces of calculation direction information include an X-axis, a Y-axis, a Z-axis, or a combination thereof in the virtual plane, the virtual three-dimensional space, or a combination thereof of the at least one object.

19. having a tolerance upon inspection in the first re-inspection step; 2. The interaction method according to claim 1, wherein, when the at least one object is a plurality of objects, the adjacent means that a pitch in a virtual plane or virtual three-dimensional space of the inspection direction of one of the at least one objects and the re-inspection direction of any one of the at least one objects is within the tolerance value.

20. the tolerance value includes a distance tolerance value and an angle tolerance value, a distance between one of the at least one object and the any one of the at least one object is within the distance tolerance value; the distance tolerance is 0-5 cm, 5-10 cm, or 10-15 cm; 20. The method of claim 19, wherein the angle tolerance is between 0 degrees and 15 degrees, or between 80 degrees and 110 degrees.

21. 1. An augmented reality interaction method for recognizing a plurality of objects in an image using a camera and a processor, comprising: Any one of the plurality of objects includes content information and a plurality of pieces of directional information, the plurality of pieces of directional information including a first direction and a second direction; The method includes steps A, B, C, and D, The step A includes: When one of the plurality of objects is adjacent to another of the plurality of objects in the first direction and the one of the plurality of objects is not adjacent to the other of the plurality of objects in the second direction, the processor generates set information corresponding to the one of the plurality of objects and stores the content information of the one of the plurality of objects in the set information; determining the one of the plurality of objects as a reference object; The step B includes: When any one of the plurality of objects other than the reference object is adjacent to the reference object in the first direction, the processor marks the any one of the plurality of objects as a follow object and executes the step C; and performing step D if the plurality of objects are not adjacent in the first direction of the reference object; Step C comprises: if the processor determines that the reference object is adjacent in the second direction of the follow object, the processor stores the content information of the follow object in the set information; determining the follow object as the reference object; and executing step B. Step D comprises: the processor processes the set information to generate a result; The interaction method according to claim 1, wherein the content information further includes relative position information, the relative position information including the plurality of directional information corresponding to the reference object of the plurality of objects.

22. 22. The method of claim 21, wherein step C further comprises: performing step D if the processor determines that the reference object is not adjacent to the follow object in the second direction.

23. Further comprising a display device, which displays a prompt image, a word image, a success prompt image, an error prompt image, a failure prompt image, a character animation image, a character skill image, or a combination thereof based on the calculation result; The interaction method according to claim 21, characterized in that step C further comprises, when the processor determines that the reference object is not adjacent to the follow object in the second direction, displaying an image corresponding to the calculation result on the display device.

24. 24. The interaction method according to claim 23, wherein in step B, if any one of the plurality of objects other than the reference object is not adjacent to the reference object in the first direction, the processor generates a prompt result and displays it on a display device.

25. 25. The method of claim 24, wherein the processor stores the content information of the reference object and the one of the plurality of objects other than the reference object in an error record set.

26. The set information includes the content information of at least one of the plurality of objects, at least one character information, a plurality of character element information in accordance with an order, a combination information of a preset character and equipment, or a combination thereof; the one of the plurality of objects is a head object, a sentence beginning object, a base object, or a character object; 24. The method of claim 23, further comprising defining any two of the plurality of objects as adjacent to each other if the shortest distance between an edge of any one of the plurality of objects and another of the plurality of objects is less than 5 cm.

27. 1. An augmented reality interaction method for recognizing at least one object in an image using a camera and a processor, comprising: The object includes content information and a plurality of pieces of calculation direction information; the plurality of pieces of calculated direction information include an inspection direction, a reinspection direction, or a combination thereof; The interaction method includes steps M, Q, and R, The step M includes: providing set information for storing the content information, a plurality of operation direction information, or a result after processing operations on the at least one object of the processor; Step Q comprises: When there are a plurality of objects in the image recognized by the processor, one of two of the plurality of objects is adjacent to the other of the two of the plurality of objects in the inspection direction, and the inspection direction of the one of the two of the plurality of objects and the re-inspection direction of the other of the two of the plurality of objects are opposite to each other, The processor includes storing the content information of the two of the plurality of objects in the collection information; The step R includes: the processor transmitting the set information to generate a computation result; the number of the at least one object is at least two, and after the step Q, a step Y is included; The step Y includes: marking said one of said two of said plurality of objects as a reference object by said processor; If another object of the plurality of objects is adjacent to the reference object in the inspection direction, the processor marks the other object of the plurality of objects as at least one follow object; and when the inspection direction of the reference object and the re-inspection direction of the at least one follow object are opposite to each other, the processor stores the content information of the reference object and the content information of at least one follow object in the set information.

28. The inspection direction includes a reading direction of an English word, a reading direction of a Chinese word, a position of an equipment slot of a character object, or a combination thereof; 28. The method of claim 27, wherein the plurality of directional information further includes a plurality of inspection directions.

29. Step X is included before step Q, 28. The method of claim 27, wherein step X comprises the processor marking one of the two of the multiple objects as a leading object if the two of the multiple objects are adjacent to one another and in the inspection direction of one of the two of the multiple objects is adjacent to the other of the two of the multiple objects and in the re-inspection direction of the one of the two of the multiple objects is not adjacent to the other of the multiple objects.

30. After the step Y, the method further includes at least one step Z, The step Z includes: marking the reference object as an inspected object by the processor, and marking the follow object as the reference object by the processor; marking another object of the plurality of objects adjacent to the reference object in the inspection direction of the reference object as another of the at least one follow object; 28. The method of interaction according to claim 27, further comprising: if the inspection direction of the reference object and the re-inspection direction of the other of the at least one follow object are opposite to each other, the processor stores the content information of the reference object and the content information of the other of the at least one follow object in the set information and executes step Z again.

31. 1. An augmented reality interaction method for recognizing at least one object in an image using a camera and a processor, comprising: The object includes content information and a plurality of pieces of calculation direction information; the plurality of pieces of calculated direction information include an inspection direction, a reinspection direction, or a combination thereof; The method includes steps P, Q′, and R, The step P includes: If the processor determines that the number of the at least one object in the image is one, the processor storing the content information of the at least one object in collection information corresponding to the object; The step Q′ includes: if the processor recognizes that the number of the at least one object in the image is multiple, and two of the plurality of objects are adjacent to each other, and in the inspection direction of one of the two of the plurality of objects, the other of the two of the plurality of objects is adjacent, and the inspection direction of the one of the two of the plurality of objects and the re-inspection direction of the other of the two of the plurality of objects are opposite to each other, the processor storing the content information of the two objects of the plurality of objects in the collection information; The step R comprises: the processor transmitting the set information to generate a computation result; the number of the at least one object is at least two, and after the step Q′, a step Y′ is included; The step Y′ is marking said one of said two of said plurality of objects as a reference object by said processor; If another object of the plurality of objects is adjacent to the reference object in the inspection direction, the processor marks the other object of the plurality of objects as at least one follow object; and when the inspection direction of the reference object and the re-inspection direction of the at least one follow object are opposite to each other, the processor stores the content information of the reference object and the content information of at least one follow object in the set information.

Images