System And Method For Controlling Physical Action Based On Multi-Modal Cross-Verification Of Target Identity

The multimodal target identity cross-verification system ensures accurate target object recognition and valid execution by combining multiple recognition methods with signed behavior tokens, preventing misrecognition and unauthorized actions.

KR103044711B1Active Publication Date: 2026-11-11GLOWFLY IDENTITY SYSTEMS CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
KR1020260142520
Authority / Receiving Office
KR · KR
Patent Type
Patents
Current Assignee / Owner
Priority Date
2025-12-22
Filing Date
2026-07-31
Publication Date
2026-11-11
Estimated Expiration
2046-07-31

AI Technical Summary

Technical Problem

Existing systems fail to reliably verify the identity of a target object before executing physical actions, leading to potential misrecognition and unauthorized actions due to discrepancies in recognition methods and execution authority verification.

Method used

A multimodal target identity cross-verification system that combines multiple recognition modalities with signed behavior tokens to ensure the identity of the target object matches the intended object, verifying the token's validity and execution constraints before allowing action commands to be transmitted to the actuator.

Benefits of technology

Prevents misrecognition and unauthorized actions by ensuring the target identity is accurately verified through multiple recognition methods, integrating token validity and execution constraints, thereby securely controlling physical actions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 112026093743615-PAT00001_ABST
    Figure 112026093743615-PAT00001_ABST
Patent Text Reader

Abstract

A physical behavior control system and method based on multimodal target identity cross-verification are disclosed. The system operates between an autonomous agent and an actuator that executes a candidate physical behavior requested by the autonomous agent, thereby controlling the transmission of a behavior command corresponding to the candidate physical behavior to the actuator. A token issuer issues a signed behavior token that includes one or more identity anchors representing the type of behavior, execution constraints, and the identity of the target object in the issuer's signature target. A target identity reader reads the identity of the target object using a plurality of recognition modalities before the execution of the candidate physical behavior is initiated, and provides a reading result for each of the plurality of recognition modalities. A token verifier verifies the signature and validity of the signed behavior token and whether the execution constraints of the behavior command are satisfied, and compares each of the reading results for each of the plurality of recognition modalities with an identity anchor designated among the one or more identity anchors to correspond to the reading result, and provides a verification result that allows the execution of the candidate physical behavior only when the conditions of the verification and comparison are satisfied. An action relayer transmits the behavior command to the actuator only when a verification result allowing execution is provided, and does not allow the transmission of the behavior command when no verification result is provided.
Need to check novelty before this filing date? Find Prior Art

Description

Technology Field

[0001] The present invention relates to a technology for controlling the execution of a physical action requested by an autonomous agent, and more specifically, to a system and method for controlling the transmission of an action command corresponding to the physical action to a driving unit using a plurality of reading results regarding the identity of a target object. Background Technology

[0002] Robots, unmanned transport devices, or other physical execution devices can perform physical actions, such as grasping or transporting target objects, manipulating equipment, or moving to specific locations, in accordance with commands generated by an autonomous agent. To safely perform these physical actions, it is necessary to verify not only whether the action command is authorized, but also whether the target object indicated by the command matches the object existing at the actual execution location.

[0003] The method of applying digital signatures to action commands or execution authority can be used to verify whether the command or authority was created by a legitimate issuer and whether its contents have not been altered. However, even if the type of action and execution conditions are validly authorized, if the autonomous agent or recognition device incorrectly identifies the actual target object, the legitimately authorized action may be executed on an unintended object.

[0004] For example, if multiple objects have similar appearances, if part of an object is obscured, if illumination or the direction of observation changes, or if disturbance occurs in the recognition input, the camera-based recognition result may differ from the identity of the actual target object. Even when using a wireless identification method, if multiple tags exist within the reading range, tags of adjacent objects that are not the target of the action may be read together. If such reading results are processed separately from the authorization for the execution of a physical action, errors in the recognition result or an incorrect correspondence between the target and the identification information may lead to actual operation.

[0005] By utilizing multiple sensors or recognition methods, a reading by one recognition method can be supplemented by another. However, if the results of multiple readings are used merely to calculate the reliability of object recognition and are not combined with the verification of signed execution rights and the transmission path of action commands, it is difficult to consider the execution of physical actions as being structurally blocked even when a discrepancy in the target identity is confirmed.

[0006] Therefore, a technology is required to verify the combination of authorization regarding the content and execution conditions of an action and the identity of the actual target object confirmed immediately before execution, and to ensure that if the combined verification result is not satisfied, the action command is not transmitted to the driving unit. Prior art literature

[0007] International Publication WO 2017 / 155598 A2 (2017.09.14) U.S. Publication US 2010 / 0115284 A1 (2010.05.06) The problem to be solved

[0008] The present invention aims to provide a multimodal target identity cross-verification-based physical behavior control system and method capable of restricting the transmission of a behavior command corresponding to a candidate physical behavior to an actuator when the identity of the actual execution target does not match the identity of the target object specified by a signed behavior token, even if the execution authority regarding the candidate physical behavior is valid.

[0009] In addition, the present invention aims to provide a multimodal target identity cross-verification-based physical behavior control system and method capable of combining a reading result immediately prior to execution by a plurality of recognition modalities with the verification of a signed behavior token, and controlling the delivery of a behavior command to an actuator according to the combined verification result.

[0010] The problems that the present invention aims to solve are not limited to those mentioned above, and other unmentioned problems can be clearly understood by a person skilled in the art from the description below. means of solving the problem

[0011] The multimodal target identity cross-verification-based physical behavior control system of the present invention is a system that operates between an autonomous agent and an actuator that executes a candidate physical behavior requested by the autonomous agent, and controls the transmission of a behavior command corresponding to the candidate physical behavior to the actuator, and comprises: a token issuing unit that issues a signed behavior token corresponding to the candidate physical behavior, the token including a type of behavior, an execution constraint applied to the execution of the candidate physical behavior, one or more identity anchors representing the identity of the target object of the candidate physical behavior, and a signature of an issuer, wherein the type of behavior, the execution constraint, and the one or more identity anchors are included in the signature of the issuer; and a target identity reading unit that reads the identity of the target object using a plurality of recognition modalities before the execution of the candidate physical behavior by the actuator is initiated, and provides a reading result for each of the plurality of recognition modalities. A token verification unit that verifies whether the signature of the above-mentioned signed action token is valid, whether the above-mentioned signed action token is valid, and whether the above-mentioned action command satisfies the above-mentioned execution constraint, and compares each of the reading results for each of the plurality of recognition modalities with an identity anchor designated to correspond to the corresponding reading result among the one or more identity anchors, and provides a verification result that allows the execution of the above-mentioned candidate physical action only when the signature is valid, the above-mentioned action token is valid, the above-mentioned action command satisfies the above-mentioned execution constraint, and the comparison result regarding whether there is a match between the reading results for each of the plurality of recognition modalities and the corresponding identity anchor satisfies a predetermined match condition;The invention is characterized by including an action relay unit coupled to the drive command transmission path between the autonomous agent and the drive unit, which transmits the action command to the drive unit only when a verification result allowing the execution of the candidate physical action is provided from the token verification unit, and does not allow the transmission of the action command when the verification result is not provided.

[0012] The physical behavior control method based on multimodal target identity cross-verification according to the present invention is performed between an autonomous agent and an actuator that executes a candidate physical behavior requested by the autonomous agent, and controls the transmission of a behavior command corresponding to the candidate physical behavior to the actuator, comprising: (a) a step in which a token issuing unit corresponds to the candidate physical behavior and issues a signed behavior token comprising a type of behavior, an execution constraint applied to the execution of the candidate physical behavior, one or more identity anchors representing the identity of the target object of the candidate physical behavior, and a signature of the issuer, wherein the type of behavior, the execution constraint, and the one or more identity anchors are included in the signature of the issuer; (b) a step in which a target identity reading unit reads the identity of the target object using a plurality of recognition modalities before the execution of the candidate physical behavior by the actuator is initiated, and provides a reading result for each of the plurality of recognition modalities; (c) A step in which a token verification unit verifies whether the signature of the signed action token is valid, whether the signed action token is valid, and whether the action command satisfies the execution constraint, and compares each of the reading results for each of the plurality of recognition modalities with an identity anchor designated to correspond to the reading result among the one or more identity anchors, and provides a verification result that allows the execution of the candidate physical action only when the signature is valid, the signed action token is valid, the action command satisfies the execution constraint, and the comparison result regarding whether there is a match between the reading results for each of the plurality of recognition modalities and the corresponding identity anchor satisfies a predetermined match condition;and (d) the action relay unit coupled to the drive command transmission path between the autonomous agent and the drive unit is characterized by including the step of transmitting the action command to the drive unit only when the verification result allowing the execution of the candidate physical action is provided, and not allowing the transmission of the action command when the verification result is not provided.; Effects of the invention

[0013] According to one aspect of the present invention, since the type of action, execution constraints, and the identity anchor of the target object are included together in the issuer's signature target, the execution authority regarding the candidate physical action and the identity of the target object on which the action is to be executed can be verified by combining them.

[0014] According to one aspect of the present invention, the identity of a target object is read using a plurality of recognition modalities before the execution of a candidate physical action is initiated, and the reading result is compared with an identity anchor included in the signature target of a signed action token, thereby preventing misrecognition or reading error caused by a single recognition modality from immediately leading to the execution of a physical action.

[0015] According to one aspect of the present invention, a verification result is provided that allows execution only when the signature and validity of a signed action token, whether the execution constraints of an action command are satisfied, and the condition of matching the target identity are all satisfied together, so that the verification of execution authority and the verification of the execution target can be combined into a single action command delivery condition.

[0016] According to one aspect of the present invention, if a verification result allowing execution is not provided, the operation relay unit does not allow the transmission of an action command, thereby blocking the path through which an action command is transmitted to the driving unit in which a discrepancy in target identity or a reading failure is confirmed.

[0017] The effects of the present invention are not limited to those mentioned above, and other unmentioned effects can be clearly understood by a person skilled in the art from the description below. Brief explanation of the drawing

[0018] FIG. 1 is a block diagram showing the overall configuration of a multimodal target identity cross-validation-based physical behavior control system according to one embodiment of the present invention. Figure 2 is a diagram showing the data structure of a signed behavior token used in the multimodal target identity cross-verification-based physical behavior control system illustrated in Figure 1, and the relationship in which an identity anchor is included in the issuer's signature target. Figure 3 is a diagram showing the relationship for determining whether a match condition is satisfied by comparing the reading results for each of the multiple recognition modalities with the corresponding identity anchor in the multimodal target identity cross-validation-based physical behavior control system illustrated in Figure 1. FIG. 4 is a flowchart illustrating a physical behavior control method performed by a multimodal target identity cross-validation-based physical behavior control system illustrated in FIG. 1. FIG. 5 is a diagram showing a configuration in which the irradiation area of ​​a directional light source and the camera-based visual recognition area are spatially aligned and the spatial conditions are verified using the response of an optically augmented RFID tag in a physical behavior control system based on multimodal target identity cross-verification shown in FIG. 1. FIG. 6 is a flowchart illustrating the process of switching to a degraded mode and re-verifying the identity of a target object during execution when some recognition modalities are unavailable in the multimodal target identity cross-verification-based physical behavior control system illustrated in FIG. 1. FIG. 7 is a diagram showing a configuration for measuring the carrier phase and estimating the position of the optically enhanced RFID tag based on the backscatter response of the optically enhanced RFID tag shown in FIG. 5. FIG. 8 is a diagram illustrating a process of identifying a target candidate, reading the tag identifier of a target object, and cross-verifying the identity and location of a target object using a first mode of the optically enhanced RFID tag and a second mode corresponding to the optical gating mode shown in FIG. 5. Specific details for implementing the invention

[0019] Hereinafter, a multimodal target identity cross-verification-based physical behavior control system and method according to an embodiment of the present invention will be described in detail with reference to the attached drawings. The following embodiments are examples to explain the technical concept of the present invention, and the scope of the present invention is not limited by the following embodiments.

[0020] FIGS. 1 through 8 illustrate a common system embodiment and optional configurations and modes of operation applied to the system. FIG. 1 shows the overall configuration of the system, FIGS. 2 through 6 show the data structure and operation of the system shown in FIG. 1, and FIGS. 7 and 8 show optional extended forms of the optically enhanced RFID reading configuration shown in FIG. 5.

[0021] In this specification, "autonomous agent" refers to an external entity that plans a physical action and requests its execution based on at least one of sensor data, predetermined rules, or the inference results of an artificial intelligence model. The autonomous agent may be implemented as robot control software, a planning module, a remote control system, or a combination thereof. In this specification, "actuator" refers to an external execution entity that actually executes an action command corresponding to a candidate physical action requested by the autonomous agent, and may include a moving mechanism, a robot arm, a gripping unit, a drone propulsion unit, an industrial actuator, or a part thereof.

[0022] In this specification, "candidate physical action" refers to a physical action requested by an autonomous agent but whose execution permission by the system of the present invention has not yet been determined. Candidate physical actions may include movement, grasping, stacking, joining, separating, opening and closing, pressurizing, discharging, or other changes in physical state. "Action command" refers to a command stream consisting of a single command or multiple commands provided over time that instructs a driving unit to execute a candidate physical action.

[0023] In this specification, "modality" refers to a recognition method or detection method for acquiring information regarding the identity of a target object. A modality may include camera-based visual recognition, barcode or QR code reading, NFC tag reading, RFID tag reading, optically enhanced RFID tag reading, shape reading by a depth sensor, or reading based on other physical principles. Since multiple sensors may be used in the same recognition method or a single sensor may be used in multiple recognition methods, the number of modalities does not necessarily correspond to the number of physical sensors.

[0024] In this specification, "identity anchor" refers to reference information included in a signed action token to identify the target object of a candidate physical action. An identity anchor may include a tag identifier, a payload of a barcode or QR code, a hash of a visual feature descriptor, a shape descriptor, a material property, or other identification information. An identity anchor may be a value that is directly compared with a read result of a specific modality, or a value that is compared after normalizing, hashing, decoding, or verifying the read result.

[0025] In this specification, "read result" includes an observation obtained from a target object using a modality, and, if necessary, the result of preprocessing or recognition processing on said observation. The read result may include an identifier, payload, feature descriptor, hash value, object classification result, location information, unreadable status, or confidence information. "Match condition" means a condition for determining that the identity of a target object is confirmed based on a comparison result between one or more read results and one or more identity anchors.

[0026] Referring to FIG. 1, a multimodal target identity cross-verification-based physical behavior control system (100) according to one embodiment of the present invention includes a token issuance unit (130), a token verification unit (140), a target identity reading unit (150), and a behavior relay unit (170). An autonomous agent (110) and a driving unit (120) are located outside the system (100), and the system (100) controls the transmission of behavior commands corresponding to candidate physical behaviors between the autonomous agent (110) and the driving unit (120).

[0027] The system (100) may be implemented as hardware including a program module executed by one or more processors, memory, a processor, and a communication interface, or a combination thereof. Two or more configurations shown separately from each other in FIG. 1 may be integrated into a single program module or hardware device, and a single configuration may be distributed among multiple program modules or hardware devices that communicate with each other. Accordingly, the functional separation of the token issuing unit (130), the token verification unit (140), the target identity reading unit (150), and the operation relay unit (170) does not necessarily mean physically separated devices.

[0028] The token issuing unit (130) issues a signed action token (220) corresponding to a candidate physical action requested by the autonomous agent (110). The token issuing unit (130) may receive request data representing a candidate physical action from the autonomous agent (110), a job management system, or a separate authorization system. The request data may include at least one of a type of action, a target object, driving unit identification information, a target location, a target attitude, a planned path, a requested physical quantity, or a job completion condition.

[0029] Referring to FIG. 2, a signed action token (220) includes an action type (221), an execution constraint (222), one or more identity anchors (228), and a signature (229) of the issuer. The action type (221) indicates a type of movement, grasping, stacking, stopping, or other candidate physical action. The execution constraint (222) may indicate at least one of a maximum speed, maximum acceleration, force, torque, travel distance, workspace, target location, allowable space area, start time, end time, or maximum duration.

[0030] The token issuing unit (130) includes a type of action (221), an execution constraint (222), and one or more identity anchors (228) in the signature target data to which the issuer's signature (229) is applied. Accordingly, if the type of action (221), the execution constraint (222), or the identity anchor (228) changes after issuance, the change can be detected during the signature verification process of the token verification unit (140). By including the identity anchor (228) in the signature target, the signed action token (220) indicates not only which action is allowed within which range, but also which target object the action is allowed for.

[0031] When multiple identity anchors (228) are used, the signed action token (220) may include two or more of a tag identifier, a barcode or QR code payload, and a hash of a visual feature descriptor. Each identity anchor (228) may further include a modality identifier or data format information representing a recognition modality corresponding to the anchor. For example, the tag identifier may correspond to an RFID read result, the QR code payload to a QR code read result, and the hash of the visual feature descriptor to a feature descriptor derived from a camera image.

[0032] According to the embodiment, the signed action token (220) may further include at least one of a token identifier, a sequence number, a valid time window, an issuance time, target driving unit identification information, and a single-use attribute. The issuer's signature (229) may be implemented as a public key digital signature or a verifiable cryptographic authentication means. The signature algorithm and serialization format are not limited to a specific method and may be applied as long as the token verification unit (140) can verify the integrity of the issuer and the data to be signed.

[0033] The target identity reading unit (150) reads the identity of the target object (10) using a plurality of recognition modalities before the execution of a candidate physical action by the driving unit (120) is initiated, and provides reading results for each of the plurality of recognition modalities. The target identity reading unit (150) of the present embodiment includes two or more of a camera (151), a barcode or QR code reader (152), an NFC reader (153), an RFID reader (154), and an optically enhanced RFID reader (155). In a variation, the target identity reading unit (150) receives reading results for each recognition modality from an external sensor device, and its function may be distributed and placed in the driving unit (120), the robot body, or a separate sensor device.

[0034] "Before execution begins" means that the reading of the target identity is completed before the change in physical state according to the candidate physical action begins, and the result is reflected in the decision of whether to allow execution. For example, in a gripping action, the reading may be performed before the gripping unit applies a gripping force to the target object (10), and in a moving action, before the driving unit (120) begins moving toward the destination. If a preparatory action or a sensor aiming action is distinct from the actual execution of the physical action, the reading may also be performed during such preparatory action.

[0035] The token verification unit (140) verifies whether the signature of the signed action token (220) is valid, whether the signed action token (220) is valid, and whether the action command satisfies the execution constraint (222). The validity of the signed action token (220) may include at least one of whether the token is not in an expired, withdrawn, consumed, or invalid state, whether it corresponds to the available time, whether it corresponds to the current candidate physical action, or whether it applies to the designated driving unit (120). Whether the execution constraint (222) of the action command is satisfied can be determined by comparing whether the speed, acceleration, force, path, position, workspace, or duration included in the action command is within the range of the execution constraint (222).

[0036] The token verification unit (140) compares each of the reading results for a plurality of recognition modalities provided by the target identity reading unit (150) with one or more identity anchors (228) designated to correspond to the reading result. The token verification unit (140) provides a verification result that allows the execution of a candidate physical action only when the signature is valid, the signed action token (220) is valid, the action command satisfies the execution constraint (222), and the comparison result regarding whether there is a match between the reading result for a recognition modality and the corresponding identity anchor (228) satisfies a predetermined match condition. If any of these conditions are not satisfied, the token verification unit (140) does not provide a verification result that allows execution.

[0037] The action relay unit (170) is coupled to the drive command transmission path (171) between the autonomous agent (110) and the drive unit (120). The action relay unit (170) transmits the action command to the drive unit (120) only when a verification result allowing the execution of a candidate physical action is provided from the token verification unit (140). If a verification result allowing execution is not provided, the action relay unit (170) discards, suspends, or blocks the action command, thereby not allowing transmission to the drive unit (120).

[0038] The action relay unit (170) can verify whether the action command and the verification result correspond to the same signed action token (220), candidate physical action, driving unit (120), or work session. If a single candidate physical action consists of multiple action commands, a single verification result may be applied to multiple action commands, or a verification result may be provided in correspondence with each action command. However, action commands for which a valid verification result has not been confirmed are not transmitted to the driving unit (120).

[0039] Referring to FIG. 3, the plurality of recognition modalities of the present embodiment include recognition modalities based on different physical principles. A camera (151) optically reads the appearance or sign of a target object (10), a barcode or QR code reader (152) reads an encoded optical pattern, and an RFID reader (154) reads a tag identifier included in a radio response. The target identity reading unit (150) reads the identity of the target object (10) using each recognition modal, but does not use the reading result by one recognition modal as an input for reading the identity of the target object (10) by another recognition modal.

[0040] Independent reading means that the reading result of one modality does not use the reading result of another modality as is as an input value, but rather that each modality generates a result regarding the target identity from its corresponding physical observation. The fact that subsequent normalization, temporal alignment, or comparison of reading results is performed in a common processing unit does not exclude independent reading.

[0041] In the embodiment of FIG. 3, the number of recognition modalities used to read the identity of a target object (10) among a plurality of recognition modalities is denoted as n, which is an integer greater than or equal to 2, and k is an integer greater than or equal to 1 and less than or equal to n. The token verification unit (140) compares each of the reading results for each of the n recognition modalities with an identity anchor (228) designated to correspond to the corresponding reading result among one or more identity anchors (228). A predetermined matching condition includes a consensus condition in which each of k or more of the reading results among the n reading results matches the identity anchor (228) designated to correspond to it.

[0042] If the target identity reading unit (150) fails to read the identity of the target object (10), the token verification unit (140) does not treat the unreadable result as a result that matches the identity anchor (228). Additionally, if two or more reading results indicate different identities of the target object, the token verification unit (140) determines that the results conflict with each other. Even if the number of matching results is k or more, if the applied policy requires a result without conflict, execution is not allowed unless the conflict is resolved.

[0043] In an embodiment using dynamic consensus conditions, the token verification unit (140) determines k and a minimum k value according to a policy corresponding to a type of behavior (221), a target object (10), a work environment, or a risk level. The reliability per recognition modality is calculated together with the reading result by each recognition modality and indicates the reliability of the reading result. The token verification unit (140) dynamically adjusts k according to at least one of illuminance, the distance between the target object (10) and the sensor used for reading each recognition modality, and the reliability per recognition modality. Even in this case, k is not adjusted to be lower than the minimum k value. n is determined as the number of recognition modalities used to read the identity of the target object (10) among a plurality of recognition modalities.

[0044] In an embodiment using a single-use token, the token verification unit (140) provides the token issuance unit (130) with a determination result regarding whether the comparison result satisfies the matching condition and whether two or more reading results conflict by indicating the identity of different target objects. The token issuance unit (130) processes the consumption of the signed action token (220) only when the matching condition is satisfied and the reading results do not conflict with each other, prior to the execution of the candidate physical action. If the matching condition is not satisfied or the reading results conflict, the token issuance unit (130) does not process the consumption of the signed action token (220).

[0045] The consumption of the token can be performed by registering the token identifier of the signed action token (220) in the consumption list, changing the token state to a consumed state, or updating the state storage to reject subsequent use of the same token identifier. The token verification unit (140) provides a verification result that allows execution only when it is confirmed from the token issuance unit (130) or the token state storage that the signed action token (220) has been consumed. Accordingly, the comparison determination and verification of the consumption state are completed before the delivery of the action command.

[0046] In a variant embodiment different from the above pre-execution consumption embodiment, an action step corresponding to a single candidate physical action may include an action command stream consisting of a plurality of action commands provided over time, and a single signed action token (220) may be verified over a plurality of verification cycles while the action step continues. In this case, the token issuer (130) may finally consume the signed action token (220) at the time when the completion report of the corresponding action step is accepted. Repeated verification of the same signed action token (220) in an active period applies to continued execution within the same action step, the action type (221) and execution constraint (222) of the corresponding token, and does not mean reusing the token for a separate action.

[0047] In a variant embodiment consumed upon completion reporting, the signed action token (220) may include at least one of a token identifier (224), a sequence number (225), an effective time window, and a chain identifier. The token issuing unit (130) or its state storage unit manages a plurality of signed action tokens (220) sharing the same chain identifier as a token chain (230), and may maintain an expected sequence number indicating a currently available sequence number for each token chain (230). One token indicated by the expected sequence number is managed in an active state, and if a token of a sequence number that has already passed or has not yet arrived is presented, the token verification unit (140) or the token issuing unit (130) does not allow execution. Accordingly, one active token is maintained for each token chain (230) without a separate lock flag, and simultaneous use of skipped sequences is restricted.

[0048] At least one of the driving unit (120), the control module managing the operation of the driving unit (120), the token verification unit (140), or the operation relay unit (170) can generate a completion report by confirming the completion of a candidate physical action. The completion report may include a chain identifier and a sequence number (225), a token identifier (224), or information corresponding thereto. The token issuing unit (130) accepts the completion report when the creator of the completion report is authenticated and the completion report corresponds to the currently active token and the expected sequence number. Multiple acceptance judgments occurring in a single action step may be attributed to the same chain identifier and sequence number (225), and the advancement of the expected sequence number and final consumption may occur only when the completion report is accepted.

[0049] When a completion report is accepted, the token issuing unit (130) monotonically increases the expected sequence number to change the token of the completed sequence number to a consumed state and does not reactivate the consumed token. When the consumed token is presented again, the token verification unit (140) or the token issuing unit (130) treats this as a consumption error or an attempt at reuse, does not allow execution, blocks the transmission of action commands, and may record this fact in the audit log. If a completion report is omitted or communication is disconnected and the valid time window passes, the token expires, is rejected in subsequent verification, and converges to a closed failure state where no verification result allowing execution is provided. If an environmental change or a fatal verification failure occurs, the token issuing unit (130) may withdraw the token chain (230). The expired or withdrawn token is not reactivated, and if a residual action is required, a new token chain (230) based on the current environment may be issued.

[0050] Referring to FIG. 4, in a physical behavior control method based on multimodal target identity cross-verification according to an embodiment of the present invention, first, a token issuing unit (130) issues a signed behavior token (220) corresponding to a candidate physical behavior (step (a); S110). The signature target of the signed behavior token (220) includes a type of behavior (221), an execution constraint (222), and one or more identity anchors (228). Accordingly, not only the type and allowable range of the candidate physical behavior but also the identity of the target object (10) on which the behavior is to be executed is provided as verifiable information associated with the issuer's signature (229).

[0051] The target identity reading unit (150) reads the identity of the target object (10) using a plurality of recognition modalities before the execution of a candidate physical action by the driving unit (120) is initiated, and provides reading results for each of the plurality of recognition modalities (step (b); S120). In this embodiment, the plurality of readings are performed sequentially according to the reading time of each modality, and whether execution is permitted is determined after the necessary reading results are obtained. Accordingly, separate from the target information at the planning time, the plurality of identity reading results obtained from the actual target object (10) at the time of execution of the candidate physical action are provided for the determination of whether execution is permitted. In a modified example, readings by two or more recognition modalities that operate independently of each other may be performed in parallel.

[0052] The token verification unit (140) verifies whether the signature and validity of the signed action token (220) and the execution constraints (222) of the action command are satisfied, and compares each of the reading results for each of the multiple recognition modalities with one or more identity anchors (228) designated to correspond to the reading result. The token verification unit (140) provides a verification result that allows the execution of a candidate physical action only when all essential verification conditions are satisfied and the comparison result regarding whether there is a match between the reading result for each recognition modality and the corresponding identity anchor (228) satisfies the match condition (step (c); S130). Accordingly, the verification of the execution authority regarding the candidate physical action and the verification of the identity of the actual target object (10) are combined into a single execution permission condition, and if either of the conditions is not satisfied, a verification result that allows execution is not provided.

[0053] When using a single-use token, step (c) includes a first process in which the token verification unit (140) provides a determination result to the token issuing unit (130) regarding whether k-of-n matching conditions are satisfied and whether two or more reading results conflict, and a second process in which a verification result is provided that allows execution after the consumption process is confirmed. After the first process, the token issuing unit (130) consumes the signed action token (220) only when the matching conditions are satisfied and two or more reading results do not conflict with each other (step (e); S140). After confirming the consumption process, the token verification unit (140) performs the second process, and the action relay unit (170) then transmits an action command (step (d); S150). If the determination result indicates that the matching conditions are satisfied and the reading results do not conflict, the first process, step (e), the second process, and step (d) are performed sequentially, and the single-use token used for the candidate physical action that satisfied the matching conditions is restricted from being reused. If the matching condition is not met or the reading results conflict, execution is not permitted because the token is not consumed and the second processing and step (d) are not performed.

[0054] The action relay unit (170) transmits an action command to the driving unit (120) only when a verification result allowing execution is provided, and does not allow the transmission of the action command when no verification result is provided (step (d); S150). Accordingly, the path for transmitting an action command to the driving unit (120) that does not satisfy the verification conditions regarding the signed action token (220), execution constraint (222), and the identity of the target object (10) is blocked. The step characters and step numbers shown in FIG. 4 are for identifying each step and do not imply that all detailed processing must be performed in the illustrated order, except where the order is determined for technical reasons. For example, the signature and validity verification of the token may be performed in some parallel with the reading of the target identity, but the transmission of the action command is performed after the necessary verification and comparison are completed.

[0055] Referring to FIG. 5, the optically enhanced RFID reader (155) of the target identity reader (150) includes a directional light source (410), a wireless reader (420), and, if necessary, a calibration unit (430). The optically enhanced RFID tag (11) is not an internal component of the system (100) but an external object coupled to the target object (10). The tag (11) provides a wireless response only when it receives a directional light signal from the directional light source (410) in optical gating mode.

[0056] In general RFID communication, a wireless reader can transmit a command to a tag by modulating a wireless carrier. In contrast, in one embodiment of the optically enhanced RFID reader (155), a directional light source (410) transmits a directional light signal modulated to include a command, and a wireless reader (420) transmits an unmodulated wireless carrier used for the backscatter response of the tag (11). The tag (11) receives a command by demodulating the directional light signal and forms a wireless response by backscatter modulating the unmodulated wireless carrier in response to the received command. The wireless reader (420) may provide the unmodulated wireless carrier so as to overlap temporally with the transmission of the command using the directional light signal, or provide the unmodulated wireless carrier during the response interval in which the tag (11) forms a wireless response after receiving the light command, and may use both methods of provision together. Therefore, the directional optical signal is not limited to a signal that simply activates the tag (11) but can be used as a communication channel to transmit commands to the tag (11), and optical gating is formed when the tag (11) that receives the optical command provides a wireless response corresponding to the command.

[0057] The directional light source (410) forms a limited illumination area in the direction where the target object (10) is located. The directional light source (410) may include an LED, a laser diode, or a steerable light source, and may be provided at an adjacent location to the robot body, the camera (151), or at the tip of the gripping part. The wireless reader (420) receives a wireless response from the tag (11) and reads the tag identifier included in the response.

[0058] The token verification unit (140) uses the reception of a radio response from an optically enhanced RFID tag (11) as a spatial condition indicating that the tag (11) is located within the illumination area of ​​a directional light source (410). Additionally, the token verification unit (140) compares the tag identifier included in the radio response with one or more identity anchors (228) of a signed action token (220) that are designated to correspond to the tag identifier. Only when the radio response is received and the tag identifier matches the designated identity anchor (228) is the corresponding optically enhanced RFID read result used as a valid read result for allowing execution. If the radio response is not received or the tag identifier does not match the designated identity anchor (228), the token verification unit (140) does not provide a verification result that allows execution.

[0059] The beam width of the directional light source can be adjusted according to the size of the irradiation area and the number of tags capable of receiving light commands. When the beam width is wide and radio responses are received from multiple tags (11) within the irradiation area, the target identity reader (150) reads a tag identifier from each radio response, and the token verification unit (140) can determine whether there is a match with the designated identity anchor (228) for each read tag identifier. The token verification unit (140) can select responses containing tag identifiers that match the designated identity anchor (228) as candidate reading results, and may not exclude candidate reading results solely on the grounds that there are responses from other tags unrelated to the designated identity anchor (228).

[0060] However, if there is no response matching the designated identity anchor (228), the token verification unit (140) does not provide a verification result that allows execution. Additionally, if a policy requires a single target reading and there are multiple matching responses, or if the spatial correspondence between the selected response and the target object (10) cannot be distinguished, the token verification unit (140) may treat this as ambiguity in reading or conflict of reading results and not provide a verification result that allows execution. The target identity reading unit (150) may further verify the spatial correspondence between the selected response and the target object (10) by using at least one of the location of the target object (10) read by the camera (151), the illumination area of ​​the directional light source, the location estimated based on the wireless response, and the re-reading with the beam width reduced. If the response of another tag causes a collision or interference or makes the correspondence with the target object (10) ambiguous, the target identity reading unit (150) may reduce the beam width and read again or provide a reading failure to the token verification unit (140).

[0061] When using a camera-based visual recognition modality, the calibration unit (430) spatially aligns the illumination area of ​​the directional light source (410) with the visual recognition area of ​​the camera (151). The calibration unit (430) stores calibration information indicating at least one of the position and orientation relationship between the directional light source (410) and the camera (151), the optical axis, the field of view, and the illumination angle, and can adjust the illumination direction of the light source or the recognition area of ​​the camera based on the information.

[0062] The token verification unit (140) uses the fact that the location of the target object (10) recognized in the visual recognition area is included in the area where the irradiation area of ​​the directional light source (410) and the visual recognition area overlap, and that the wireless response of the tag (11) is received while the directional light source (410) irradiates the overlap area, to verify the reading result regarding the identity of the target object (10). Accordingly, an observable relationship in which the visually recognized target object (10) and the wirelessly identified tag (11) are located in the same overlap area is reflected in the verification.

[0063] In an embodiment using a nonce per authorization session, the target identity reader (150) generates or receives a nonce for each authorization session in which verification of the signed action token (220) is performed, and modulates a directional optical signal to include the nonce. The wireless response of the tag (11) includes a response value generated by applying a predetermined response function to the nonce received through the directional optical signal. The predetermined response function may be implemented as a function that outputs the nonce as is, a function that applies a predetermined operation to the nonce, or a function that generates an authentication value based on the nonce and the secret information of the tag.

[0064] An echo-type response function that outputs a nonce as is can be implemented by extending the command-response structure of a tag chip, in which the tag (11) receives a nonce included in the payload of an optical command and returns the nonce as a response value, and the tag chip demodulates the optical command to receive the payload and returns a response corresponding to the command. A response function that applies a predetermined operation to the nonce can be selectively implemented depending on the complexity of the operation and the processing capability of the tag (11). A response function that generates an authentication value based on the nonce and the secret information of the tag (11) is a modified embodiment that can be implemented in a tag chip including a key storage unit that stores secret information and cryptographic operation logic. The fact that the echo-type response function is within the scope of being implementable as an extension of the current command-response structure does not mean that the function has already been fully implemented in a specific tag firmware.

[0065] The token verification unit (140) calculates an expected response value by applying the same response function to the nonce of the current authorization session, and uses the radio response as a valid read result used to verify the signed action token (220) of the current authorization session only if the response value of the tag (11) matches the expected response value and the radio response is received in the current authorization session. Accordingly, it is possible to restrict past responses obtained in other authorization sessions from being used as read results of the current session.

[0066] In a method using an optically enhanced RFID reading modality, step (b) includes the target identity reader (150) reading the identity of the target object (10) using a camera-based visual recognition modality and an optically enhanced RFID tag reading modality. The target identity reader (150) reads the radio response provided by the optically enhanced RFID tag (11) only when it receives a directional light signal in optical gating mode, and spatially aligns the illumination area of ​​the directional light source (410) with the visual recognition area of ​​the camera (151). Step (c) includes the token verification unit (140) using the reception of the radio response as a spatial condition indicating that the tag (11) is located in the illumination area, and providing a verification result that allows execution only when the tag identifier of the radio response matches one or more identity anchors (228) designated to correspond to the tag identifier. If the radio response is not received or the tag identifier does not match the designated identity anchor (228), the token verification unit (140) does not provide a verification result that allows execution. Accordingly, the spatial correspondence between the target object (10) recognized in the visual recognition area and the tag (11) that provided the wireless response, as well as whether it matches the designated identity anchor (228), are reflected in whether execution is allowed.

[0067] In an embodiment of spoofing determination, the token verification unit (140) monitors whether a wireless response containing a tag identifier matching an identity anchor (228) designated to correspond to a tag identifier is received while the directional light source (410) in the optical gating mode is not transmitting a directional light signal. Since the tag (11) in the normal optical gating mode does not respond if it does not receive a directional light signal, a response with a matching identifier received during the non-transmission of the light signal may be a response by a cloned tag or by bypassing optical gating. In this case, the token verification unit (140) determines the response as a spoofing attempt and does not provide a verification result that allows execution. This monitoring is applied to the optical gating query section of the second mode. In the query section of the first mode, where the tag responds to the wireless query even without light, the response during the non-transmission of light may be a normal response; therefore, the token verification unit (140) does not use the response itself of the first mode as the basis for the above spoofing determination. The target identity reading unit (150) or the token verification unit (140) can distinguish between a first mode query section and a second mode query section based on at least one of the mode switching time, the type of query command, or query session identification information.

[0068] Referring to FIG. 6, if some of the multiple recognition modalities fail to provide a reading result or if the reliability calculated together with the reading result is less than a threshold value set by a policy corresponding to a type of behavior (221), the token verification unit (140) checks whether the policy allows execution in a performance degradation mode. Failure to provide a reading result may occur due to sensor failure, out of the reading range, insufficient illumination, no tag attached, or communication failure.

[0069] If the policy does not allow a performance degradation mode, the token verification unit (140) does not provide a verification result that allows execution. If the policy allows a performance degradation mode, the token verification unit (140) determines one or more re-verification points to re-read the identity of the target object (10) during execution, the enhanced matching condition applied to the reading result of the available recognition modality, the reduced driving parameter with an upper limit set lower than at least one upper limit of the speed, force, torque, and travel distance allowed in normal mode.

[0070] The enhanced match condition requires a greater number of matched readouts than in normal mode, or a higher ratio of the number of matched readouts to the number of recognition modalities used, or requires a match of all available recognition modalities. The reduced drive parameter is a value in which at least one allowable upper limit of speed, force, torque, and travel distance is set lower than the corresponding upper limit of normal mode. The re-verification point is determined by at least one of time, travel distance, action step, and distance to the target object (10).

[0071] The token verification unit (140) provides a verification result that allows execution only when the action command satisfies the reduced driving parameters and the enhanced match condition is satisfied. The target identity reading unit (150) re-reads the identity of the target object (10) using the available recognition modalities at one or more re-verification points. The token verification unit (140) continues to provide the verification result only while the re-verification result satisfies the enhanced match condition, and stops providing the verification result when the condition is not satisfied. The action relay unit (170) stops the transmission of the action command when the provision of the verification result is stopped.

[0072] The token verification unit (140) determines the recognition modality to be used for the enhanced matching condition among the available recognition modalities according to the priority set by the policy. For example, the priority may be set in the order of optically enhanced RFID reading, barcode or QR code reading, and NFC reading, or a different order may be applied depending on the type of action and reading distance. The performance degradation mode is a mode for continuing execution within the enhanced alternative condition and reduced operating range only for actions allowed by the policy, rather than unconditionally allowing a lower safety standard than the normal mode.

[0073] Referring to FIG. 7, a wireless reader (420) transmits an unmodulated wireless carrier for backscatter. An optically enhanced RFID tag (11) forms a radio response by backscatter modulating the unmodulated wireless carrier in response to a command received via a directional optical signal. The wireless reader (420) receives and demodulates the formed radio response to obtain a tag identifier or other response information. The "radio carrier" in FIG. 7 represents such an unmodulated wireless carrier, and the label itself in the drawing has been retained for simplicity.

[0074] In the position estimation embodiment illustrated in FIG. 7, the target identity reading unit (150) further includes a phase measuring unit (440) for measuring the carrier phase of the wireless response. The phase measuring unit (440) obtains phase measurements at a plurality of observation points during the relative movement of the target object (10) and the wireless reader (420). The relative movement may be formed by the movement of a robot equipped with the wireless reader (420), the movement of a robot arm, the movement of the target object (10), or a combination thereof.

[0075] The phase measurement unit (440) estimates the position of the optically enhanced RFID tag (11) based on phase measurements from multiple observation points. In another embodiment, if the wireless reader (420) includes multiple antennas (421, 422), the phase measurement unit (440) can estimate the position of the tag (11) based on phase measurements obtained from the multiple antennas (421, 422). Position estimation may utilize at least one of a phase difference, a change in phase by distance, an angle of arrival, or a synthetic aperture processing.

[0076] The token verification unit (140) uses the estimated location of the optically enhanced RFID tag (11) as location information of a spatial condition indicating that the tag (11) is located in the irradiation area of ​​the directional light source (410). In this case, in addition to confirming whether the location is within the irradiation area by whether a wireless response is received, the estimated coordinates or area of ​​the tag (11) are included in the reading result.

[0077] In an embodiment using spatial constraints, the execution constraint (222) includes a spatial constraint representing the location or spatial area of ​​the target object (10) where the execution of the candidate physical action is allowed. The token verification unit (140) uses the location of the estimated tag (11) as location information representing the location of the target object (10) to further verify whether the location information satisfies the spatial constraint, and determines whether the execution of the candidate physical action is allowed based on the verification. If the estimated location is within the allowed work area or within the allowable error range from the target location, the token verification unit (140) determines that the spatial constraint is satisfied.

[0078] In the dual-mode embodiment illustrated in FIG. 8, the optically enhanced RFID tag (11) supports a first mode that responds to a wireless query even when no directional optical signal is received, and a second mode that corresponds to an optical gating mode and responds wirelessly only when a directional optical signal is received. In one embodiment, the first mode and the second mode are implemented together in a single optically enhanced RFID tag chip and can be selected or switched according to a read request or query mode. The software settings on the read side can select a command channel to be used for the query among a general RF command channel and a directional optical command channel, and a query section where the general RF command channel is selected can be operated as a first mode query section, and a query section where the directional optical command channel is selected can be operated as a second mode query section. The tag (11) is not limited to a method of pre-setting a separate mode state internally, but can provide a corresponding wireless response according to the command channel of the query actually received. However, the implementation of dual mode is not limited to a single chip or the above command channel selection method, and the tag configuration combined with the target object (10) can be implemented to functionally support the first mode and the second mode.

[0079] The target identity reading unit (150) identifies the tag identifier and first location information of a target object (10) among a plurality of target candidates using a wireless response received in the first mode. The first location information is calculated using at least one of the response strength, phase, reading antenna, or pre-stored location information, and is used to determine a candidate among the plurality of target candidates to aim the directional light source (410).

[0080] When a target candidate is identified, the target identity reading unit (150) sets the irradiation area of ​​the directional light source (410) to include the location indicated by the first location information and reads the tag identifier of the target object (10) from the radio response of the second mode. In the second mode, only the tag (11) that received the directional light signal responds, so the tag identifier associated with the targeted target object (10) can be read. The token verification unit (140) uses the fact that the tag identifiers read in the first mode and the second mode match each other, and that the radio response of the second mode was received while irradiating the irradiation area set based on the first location information, for cross-verification of the identity and location of the target object (10).

[0081] The first and second modes may be read by a single antenna switching in time, or distributed among multiple antennas. In a variant using multiple antennas, the first antenna may read the first mode response and the second antenna may read the second mode response, and it may be possible to compare whether the same tag identifier is read in the two response streams. This number of antennas and division of roles are optional implementations, and stepwise target verification by dual mode is not limited to a specific antenna configuration.

[0082] In another variation, the identity anchor (228) may include a three-dimensional shape descriptor or material reflection characteristic dependent on the time of observation. The target identity reader (150) may generate a reading result using a recognition modality corresponding to this identity anchor (228).

[0083] In another variant, the tag may include a long-range wireless interface and a near-range wireless interface along with a shared memory. The target identity reader (150) can identify a target candidate using the long-range interface, and after the driving unit (120) accesses the target object (10), the target identity can be reconfirmed by reading the identification information of the same shared memory using the near-range interface.

[0084] In another variation, a marker device coupled to the target object (10) can output a visible marker, such as an LED, in response to a wireless command. The target identity reader (150) can determine the identity and spatial correspondence of the target object (10) by combining a wirelessly assigned identifier with the location of the visible marker observed in the visual recognition area of ​​the camera (151).

[0085] When multiple systems or multiple robots read the same target object (10), the reading result of each system may be provided to a token verification unit (140) or a separate adjustment device and used for cross-verification between systems. In addition, the target identity cross-verification of the present invention may be applied not only to the grasping or transport of an object but also to the identification of a follower target, the verification of an entry / exit target, or the verification of an equipment operation target.

[0086] In the embodiments described above, each functional part may be implemented in hardware, software, firmware, or a combination thereof. The integration or separation of each functional part, communication method, data representation, type and number of modalities, and the order of execution of detailed steps may be changed without departing from the technical spirit of the present invention.

[0087] The embodiments described above are illustrative of the technical concept of the present invention. A person skilled in the art to which the present invention pertains may make various changes and modifications within the scope of the technical concept of the present invention without departing from it. Explanation of the symbols

[0088] 100: Multimodal Target Identity Cross-Verification-Based Physical Behavior Control System 10: Target object 11: Optically enhanced RFID tag 110: Autonomous Agent 120: Drive Unit 130: Token Issuance Unit 140: Token Verification Unit 150: Target identity reader 151: Camera 152: Barcode or QR code reader 153: NFC reader 154: RFID reader 155: Optically enhanced RFID reader 170: Operation relay unit 171: Driving command transmission path 220: Signed action token 221: Action type 222: Execution constraint 224: Token identifier 225: No. 228: Identity Anchor 229: Issuer's signature 230: Token chain 410: Directional light source 420: Radio reader 421, 422: Antenna 430: Calibration section 440: Phase measurement unit

Claims

Claim 1 A multimodal target identity cross-verification-based physical behavior control system that operates between an autonomous agent and an actuator that executes a candidate physical behavior requested by the autonomous agent, and controls the transmission of a behavior command corresponding to the candidate physical behavior to the actuator, comprising: a token issuing unit that issues a signed behavior token corresponding to the candidate physical behavior, comprising a type of behavior, an execution constraint applied to the execution of the candidate physical behavior, one or more identity anchors representing the identity of the target object of the candidate physical behavior, and a signature of an issuer, wherein the type of behavior, the execution constraint, and the one or more identity anchors are included in the signature of the issuer; a target identity reading unit that reads the identity of the target object using a plurality of recognition modalities before the execution of the candidate physical behavior by the actuator is initiated, and provides a reading result for each of the plurality of recognition modalities; and a target identity reading unit that verifies whether the signature of the signed behavior token is valid, whether the signed behavior token is valid, and whether the behavior command satisfies the execution constraint, and compares each of the reading results for each of the plurality of recognition modalities with an identity anchor designated to correspond to the corresponding reading result among the one or more identity anchors, wherein the signature is valid and the signed behavior A token verification unit that provides a verification result allowing the execution of the candidate physical action only when the token is valid, the action command satisfies the execution constraint, and the comparison result regarding whether there is a match between the reading result for each of the plurality of recognition modalities and the corresponding identity anchor satisfies a predetermined match condition;and includes an action relay unit coupled to the drive command transmission path between the autonomous agent and the drive unit, which transmits the action command to the drive unit only when a verification result allowing the execution of the candidate physical action is provided from the token verification unit, and does not allow the transmission of the action command when the verification result is not provided; the plurality of recognition modalities includes recognition modalities based on different physical principles; the target identity reading unit reads the identity of the target object using each of the recognition modalities, but does not use the reading result by one recognition modality as an input for reading the identity of the target object by another recognition modality; the number of recognition modalities among the plurality of recognition modalities used to read the identity of the target object is denoted as n, an integer greater than or equal to 2, and k is an integer greater than or equal to 1 and less than or equal to n; the matching condition includes a consensus condition in which each of the k or more reading results among the n recognition modalities matches an identity anchor designated to correspond to the corresponding reading result among the one or more identity anchors; and the token verification unit corresponds a reading result that cannot read the identity of the target object to the corresponding reading result. A multimodal target identity cross-validation-based physical behavior control system that does not treat reading results matching a designated identity anchor. Claim 2 delete Claim 3 delete Claim 4 A multimodal target identity cross-verification-based physical behavior control system according to claim 1, wherein the reliability per recognition modality represents the reliability of the reading result calculated together with the reading result by each recognition modality, and the token verification unit determines the k and minimum k values ​​according to a policy, and dynamically adjusts the k according to at least one of illuminance, the distance between the target object and the sensor used for reading each recognition modality, and the reliability per recognition modality, but does not adjust the k to be lower than the minimum k value. Claim 5 A multimodal target identity cross-validation-based physical behavior control system according to claim 1, wherein the one or more identity anchors include two or more of a tag identifier, a payload of a barcode or QR code, and a hash of a visual feature descriptor, and the target identity reader reads the identity of the target object using different recognition modalities designated to correspond to each of the two or more identity anchors among the plurality of recognition modalities. Claim 6 A multimodal target identity cross-verification-based physical behavior control system according to claim 1, wherein the signed behavior token is a single-use token, the token verification unit provides the token issuing unit with a determination result regarding whether the comparison result satisfies the matching condition and whether two or more reading results by the plurality of recognition modalities conflict by indicating the identity of different target objects, and the token issuing unit consumes the signed behavior token only when the determination result indicates that the matching condition is satisfied and the two or more reading results do not conflict with each other, and does not consume the signed behavior token when the determination result indicates that the matching condition is not satisfied or the two or more reading results conflict, and the token verification unit provides the verification result allowing the execution of the candidate physical behavior only when it is confirmed that the signed behavior token has been consumed. Claim 7 In claim 1, if some of the plurality of recognition modalities fail to provide a reading result or if the reliability calculated together with the reading result is less than a threshold value determined by a policy corresponding to the type of action, the token verification unit determines one or more re-verification points for re-reading the identity of the target object during the execution of the candidate physical action, determined by at least one of time, distance, action step, and distance to the target object, and only when the policy allows execution in a performance degradation mode, an enhanced matching condition that requires a greater number of matching reading results than in normal mode or a higher ratio of the number of matching reading results to the number of recognition modalities used, or requires a matching of all the available recognition modalities, reduced driving parameters in which the upper limit is set lower than at least one upper limit among speed, force, torque, and distance traveled allowed in normal mode, and at least one of the following: time, distance traveled, action step, and distance to the target object; the token verification unit provides the verification result that allows the execution of the candidate physical action only when the action command satisfies the reduced driving parameters and the enhanced matching condition is satisfied, and the target identity reading unit at the one or more re-verification points. A multimodal target identity cross-verification-based physical behavior control system that re-reads the identity of the target object using the available recognition modalities, and the token verification unit continues to provide the verification result only while the reading result at one or more re-verification points satisfies the enhanced match condition, and stops providing the verification result if the enhanced match condition is not satisfied, and the action relay unit stops the transmission of the action command when the provision of the verification result is stopped, and the token verification unit determines the recognition modal among the available recognition modalities to be used for the enhanced match condition according to the priority determined by the policy. Claim 8 A multimodal target identity cross-verification-based physical behavior control system according to claim 1, wherein the target identity reading unit includes a directional light source and a wireless reader, and one of the plurality of recognition modalities is an optically enhanced RFID tag reading modal that reads a wireless response provided only when an optically enhanced RFID tag coupled to the target object receives a directional light signal from the directional light source in an optical gating mode using the directional light source and the wireless reader, and the token verification unit uses the reception of the wireless response from the optically enhanced RFID tag as a spatial condition indicating that the optically enhanced RFID tag is located in the irradiation area of ​​the directional light source, and provides the verification result that allows the execution of the candidate physical behavior only when the tag identifier included in the wireless response matches an identity anchor designated to correspond to the tag identifier among the one or more identity anchors, and does not provide the verification result when the wireless response is not received or the tag identifier does not match the designated identity anchor. Claim 9 In claim 8, the plurality of recognition modalities further include a camera-based visual recognition modal, the target identity reading unit further includes a calibration unit that spatially aligns the irradiation area of ​​the directional light source with the visual recognition area of ​​the camera-based visual recognition modal, and the token verification unit uses the fact that the location of the target object recognized in the visual recognition area is included in the area where the irradiation area of ​​the directional light source and the visual recognition area overlap, and the wireless response received while the directional light source irradiates the overlapping area, to verify the reading result for the identity of the target object in a multimodal target identity cross-verification-based physical behavior control system. Claim 10 A multimodal target identity cross-verification-based physical behavior control system according to claim 8, wherein the target identity reading unit modulates the directional optical signal to include a nonce per authorization session in which verification of the signed behavior token is performed, and the radio response received from the optically enhanced RFID tag includes a response value generated by applying a predetermined response function to the nonce received through the directional optical signal, and the token verification unit calculates an expected response value by applying the predetermined response function to the nonce per authorization session, and uses the radio response as a valid reading result used for verification of the signed behavior token performed in the authorization session only when the response value matches the expected response value and the radio response is received in the authorization session. Claim 11 A multimodal target identity cross-verification-based physical behavior control system according to claim 8, wherein the token verification unit determines the wireless response as a spoofing attempt by clone tag or optical gating bypass and does not provide the verification result that allows the execution of the candidate physical behavior when a wireless response including a tag identifier matching an identity anchor designated to correspond to the tag identifier among one or more identity anchors is received while the directional light source does not transmit the directional light signal in the optical gating mode. Claim 12 A multimodal target identity cross-verification-based physical behavior control system according to claim 8, wherein the wireless reader transmits a wireless carrier for backscatter, and the wireless reader receives and demodulates the wireless response formed by backscatter modulating the wireless carrier in response to a command received by the optically enhanced RFID tag through the directional optical signal. Claim 13 In claim 12, the target identity reading unit further includes a phase measuring unit that measures the carrier phase of the wireless response, and the phase measuring unit estimates the position of the optically enhanced RFID tag based on at least one of a phase measuring value obtained at a plurality of observation points during relative movement between the target object and the wireless reader and a phase measuring value obtained from the plurality of antennas when the wireless reader includes a plurality of antennas, and the token verification unit uses the estimated position of the optically enhanced RFID tag as position information representing the spatial condition in a multimodal target identity cross-verification-based physical behavior control system. Claim 14 A multimodal target identity cross-validation-based physical behavior control system according to claim 13, wherein the execution constraint includes a spatial constraint representing a location or spatial region of the target object where the execution of the candidate physical behavior is permitted, and the token verification unit further verifies whether the location information satisfies the spatial constraint by using the estimated location of the optically enhanced RFID tag as location information representing the location of the target object, and determines whether the execution of the candidate physical behavior is permitted based on the verification. Claim 15 In claim 8, the optically enhanced RFID tag coupled to the target object supports a first mode that responds to a wireless query even when the directional light signal is not received, and a second mode that corresponds to the optical gating mode and responds wirelessly only when the directional light signal is received; the target identity reading unit identifies the tag identifier and first location information of the target object among a plurality of target candidates using the wireless response received in the first mode, sets the irradiation area of ​​the directional light source to include the location indicated by the first location information, reads the tag identifier of the target object from the wireless response received in the second mode, and the token verification unit uses the fact that the tag identifiers read in the first mode and the second mode match each other and the wireless response of the second mode is received while irradiating the irradiation area set based on the first location information for cross-verification of the identity and location of the target object, in a multimodal target identity cross-verification-based physical behavior control system. Claim 16 A multimodal target identity cross-verification-based physical action control method performed between an autonomous agent and an actuator that executes a candidate physical action requested by the autonomous agent, wherein the method controls the transmission of an action command corresponding to the candidate physical action to the actuator, comprising: (a) a step in which a token issuing unit issues a signed action token corresponding to the candidate physical action, the action type, an execution constraint applied to the execution of the candidate physical action, one or more identity anchors representing the identity of the target object of the candidate physical action, and a signature of the issuer, wherein the action type, the execution constraint, and the one or more identity anchors are included in the signature of the issuer; (b) a step in which a target identity reading unit reads the identity of the target object using a plurality of recognition modalities before the execution of the candidate physical action by the actuator is initiated, and provides a reading result for each of the plurality of recognition modalities; (c) a token verification unit verifies whether the signature of the signed action token is valid, whether the signed action token is valid, and whether the action command satisfies the execution constraint, and sets each of the reading results for each of the plurality of recognition modalities to correspond to the corresponding reading result among the one or more identity anchors. A step of providing a verification result that allows the execution of the candidate physical action only when, by comparing with a designated identity anchor, the signature is valid, the signed action token is valid, the action command satisfies the execution constraint, and the comparison result regarding whether there is a match between the reading result for each of the plurality of recognition modalities and the corresponding identity anchor satisfies a predetermined match condition;and (d) a motion relay unit coupled to the drive command transmission path between the autonomous agent and the drive unit transmits the action command to the drive unit only when the verification result allowing the execution of the candidate physical action is provided, and does not allow the transmission of the action command when the verification result is not provided, wherein step (b) comprises the target identity reading unit reading the identity of the target object using each of n recognition modalities based on different physical principles, wherein the reading result by one recognition modality is not used as an input for reading the identity of the target object by another recognition modality, and n is an integer of 2 or greater, which is the number of recognition modalities used to read the identity of the target object among the plurality of recognition modalities, and step (c) comprises the token verification unit applying k, an integer of 1 or greater and n or less, to apply the matching condition that each of the k or more reading results among the n recognition modalities matches an identity anchor designated to correspond to the corresponding reading result among the one or more identity anchors, and the reading result for which the identity of the target object cannot be read is the identity designated to correspond to the corresponding reading result A physical behavior control method based on multimodal target identity cross-validation that is not treated as a reading result matching an anchor.; Claim 17 In claim 16, the above step (a) comprises the token issuing unit issuing the signed action token, which is a single-use token; the above step (c) comprises the first processing of the token verification unit providing the token issuing unit with a determination result regarding whether two or more reading results among the n recognition modalities conflict by indicating the identity of different target objects and whether the contrast result satisfies the matching condition, and the second processing of providing the verification result that allows the execution of the candidate physical action only when the consumption processing of the signed action token is confirmed; and (e) further comprises the step of the token issuing unit consuming the signed action token only when the determination result indicates that the matching condition is satisfied and the two or more reading results do not conflict with each other, and not consuming the signed action token when the determination result indicates that the matching condition is not satisfied or the two or more reading results conflict, wherein the method for controlling physical action based on multimodal target identity cross-verification is performed in the order of the first processing, the above step (e), the above second processing, and the above step (d). Claim 18 In claim 16, the above step (b) is a method for controlling physical behavior based on multimodal target identity cross-verification, wherein the target identity reading unit reads the identity of the target object using the plurality of recognition modalities including a camera-based visual recognition modality and an optically enhanced RFID tag reading modality, wherein the optically enhanced RFID tag reading modality reads a radio response provided only when the optically enhanced RFID tag coupled to the target object receives a directional light signal in optical gating mode using a directional light source and a radio reader, and spatially aligns the illumination area of ​​the directional light source with the visual recognition area of ​​the camera-based visual recognition modal, and the above step (c) is a method for controlling physical behavior based on multimodal target identity cross-verification, wherein the token verification unit uses the reception of the radio response from the optically enhanced RFID tag as a spatial condition indicating that the optically enhanced RFID tag is located in the illumination area of ​​the directional light source, and provides a verification result that allows the execution of the candidate physical behavior only when the tag identifier included in the radio response matches an identity anchor designated to correspond to the tag identifier among the one or more identity anchors, and does not provide the verification result when the radio response is not received or the tag identifier does not match the designated identity anchor.

Citation Information

Patent Citations

  • Target object detection apparatus and robot provided with the same

    JP2006192563A

  • Remote control method and apparatus

    US20240313960A1

  • Utilizing Blockchain Technology with Mobile Robotic Machines

    US20200242538A1

  • Method for controlling commands suitable to be processed by a peripheral such as an actuator

    US20210011871A1

  • Method to support autonomous robot-to-robot identification and task transfer

    US20220107625A1