Intelligent agent positioning execution control method and device, equipment, medium and product

Abstract

The embodiment of the invention provides an intelligent agent positioning execution control method which can be applied to the technical field of artificial intelligence. The intelligent agent positioning execution control method comprises the following steps: determining a touch detection range of a detection end of an intelligent agent according to a received initial visual image; controlling the detection end to execute touch detection in the touch detection range so as to generate a touch detection image; the touch detection process of the detection end is corrected through the contact detection image, a contact positioning image is generated, and the contact positioning image is used for controlling an execution end of the intelligent agent to conduct positioning execution. The embodiment of the invention further provides an intelligent agent positioning execution control device and equipment, a storage medium and a program product.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, specifically to the field of intelligent body control technology, and more specifically to a positioning execution control method, device, equipment, medium and product for an intelligent body. Background Art

[0002] Artificial Intelligence (AI) is an important driving force for the new round of scientific and technological revolution and industrial transformation. It is a new key technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. As an important part of intelligent science, artificial intelligence attempts to understand the essence of intelligence and produce a new intelligent machine (i.e., intelligent agent) that can respond in a similar way to human intelligence.

[0003] For embodied intelligence, the precise positioning operation of complex action execution is one of the important aspects that reflect the level of intelligence. For example, embodied intelligence picks up tiny objects on the ground (such as steel nails), imitates human unlocking and inserting keys into keyholes, etc. The intelligent agent must be able to accurately perceive the positions of steel nails, keyholes, etc., and then perform action positioning execution to achieve precise operations. For intelligent agents, visual and tactile perception, as "eyes" and "skin", are important sensory functions when intelligent agents interact with the external environment, and are one of the key technologies for intelligent robot operation. Visual information can provide robots with information about the surrounding environment, the number and distribution of existing objects, and the outline and size of each object; tactile information can provide robots with information about the surface characteristics of objects, the contact status between robots and objects, etc.

[0004] In order to obtain more accurate positioning operations, the existing technology usually uses high-cost detection equipment or perception sensors (such as high-end cameras or radars, etc.), which makes the perception cost of visual and tactile information in traditional technology high. In addition, the use of such expensive detection equipment and perception sensors requires a set of systematic software systems to match them, which further increases the amount of data required, resulting in poor versatility. The data processing method is often not universal enough, and each intelligent agent can only use it exclusively. Summary of the invention

[0005] In view of at least one of the above problems, the embodiments of the present invention aim to provide a positioning execution control method, device, equipment, medium and product for an intelligent body with high precision, which has more universal data processing and simpler operation on the basis of lower cost, thereby providing a method that can achieve high-precision measurement of the six-dimensional information of the target object in space by using low-cost detection equipment such as tactile sensors and detection cameras, and can make full use of visual and tactile perception feedback to improve positioning efficiency and operational performance, while being able to achieve rapid adaptation across different intelligent body systems, thereby maintaining a higher level of intelligence.

[0006] One aspect of an embodiment of the present invention provides a positioning execution control method for an intelligent agent, which includes: determining the tactile detection range of a detection end of the intelligent agent based on a received initial visual image; controlling the detection end to perform touch detection within the tactile detection range to generate a contact detection image; and correcting the touch detection process of the detection end through the contact detection image to generate a contact positioning image, and the contact positioning image is used to control the execution end of the intelligent agent to perform positioning execution.

[0007] According to an embodiment of the present invention, before determining the tactile detection range of the detection end of the intelligent agent according to the received initial visual image, it also includes: performing visual detection on the target detection space where the intelligent agent is located to generate an initial visual image.

[0008] According to an embodiment of the present invention, determining the tactile detection range of the detection end of the intelligent agent according to the received initial visual image includes: performing object position resolution on the initial visual image according to the target execution object; and determining the tactile detection range according to the object position resolution.

[0009] According to an embodiment of the present invention, controlling the detection end to perform touch detection within the tactile detection range to generate a contact detection image includes: acquiring a touch detection execution position within the tactile detection range; and controlling the detection end to perform touch detection on the touch detection execution position to generate a contact detection image.

[0010] According to one embodiment of the present invention, in the process of correcting the touch detection process of the detection end through the contact detection image to generate the contact positioning image, it includes: obtaining plane position correction data and posture angle correction data through the contact detection image; updating the touch detection execution position of the detection end within the tactile detection range according to the plane position correction data; and adjusting the touch detection angle of the detection end when performing touch detection at the updated touch detection execution position according to the posture angle correction data; and controlling the detection end to perform touch detection on the updated touch detection execution position according to the touch detection angle until the contact positioning image is generated.

[0011] According to an embodiment of the present invention, before acquiring the plane position correction data and the posture angle correction data through the contact detection image, the method further includes: identifying a target imaging area of ​​the target execution object in the contact detection image.

[0012] According to an embodiment of the present invention, acquiring the plane position correction data through the contact detection image includes: comparing the target imaging area with the imaging field of view area of ​​the contact detection image to acquire the plane position correction data.

[0013] According to one embodiment of the present invention, in obtaining posture angle correction data through contact detection images, it includes: extracting a filtered contact image of the contact detection image; obtaining the correspondence between each pixel point on the filtered contact image and a preset simulated contact image, and obtaining posture angle correction data.

[0014] Another aspect of an embodiment of the present invention provides a positioning execution control device of an intelligent body, which includes a range determination module, an image generation module and a touch correction module. The range determination module is used to determine the tactile detection range of the detection end of the intelligent body according to the received initial visual image; the image generation module is used to control the detection end to perform touch detection within the tactile detection range to generate a contact detection image; and the touch correction module is used to correct the touch detection process of the detection end through the contact detection image to generate a contact positioning image, and the contact positioning image is used to control the execution end of the intelligent body to perform positioning execution.

[0015] Another aspect of an embodiment of the present invention provides an electronic device, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above-mentioned intelligent body positioning execution control method.

[0016] Another aspect of an embodiment of the present invention provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the above-mentioned positioning execution control method of the intelligent body.

[0017] Another aspect of an embodiment of the present invention provides a computer program product, including a computer program, which implements the positioning execution control method of the above-mentioned intelligent agent when executed by a processor.

[0018] The positioning execution control method of an intelligent body provided by an embodiment of the present invention can at least partially solve at least one of the technical problems existing in the related art, and thus can achieve at least one of the following technical effects: First of all, only lower-cost visual and tactile sensing devices can be used to achieve high-precision positioning and execution operations. The installation and calibration of these sensing devices are simpler and more convenient, and there is no need for frequent setup operations for different objects and scenes, saving time and effort.

[0019] Secondly, the processing of positioning and executing operations based on visual and tactile information no longer needs to strictly rely on end-to-end processing based on neural networks, which makes it possible to achieve higher versatility for different scenarios and different execution tasks, and no longer requires higher data volumes in traditional methods.

[0020] Therefore, even low-cost tactile sensors, detection cameras and other detection equipment can be used to achieve high-precision measurement of the six-dimensional information of the target object in space, and visual and tactile perception feedback can be fully utilized to improve positioning efficiency and operational performance. At the same time, it can achieve rapid adaptation across different intelligent body systems, has higher versatility, a wider range of applications, and can maintain a higher level of intelligence.

[0021] It should be understood that the above general description and the following detailed description are merely exemplary and illustrative and are not intended to limit the scope of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which: Figure 1 A schematic diagram showing an application scenario of a positioning execution control method, apparatus, device, medium and program product of an intelligent body according to an embodiment of the present invention; Figure 2 A flowchart of a positioning execution control method of an intelligent body according to an embodiment of the present invention is schematically shown; Figure 3 A flowchart of an application scenario of the positioning execution control method of an intelligent body according to an embodiment of the present invention is schematically shown; Figure 4A A schematic diagram showing an application scenario of a plane position correction of a positioning execution control method of an intelligent body according to an embodiment of the present invention; Figure 4B A diagram schematically shows an application scenario of posture angle correction of a positioning execution control method of an intelligent body according to an embodiment of the present invention; Figure 5 A schematic diagram showing a structural block diagram of a positioning execution control device for an intelligent body according to an embodiment of the present invention; and Figure 6 A block diagram of an electronic device suitable for implementing a positioning execution control method for an intelligent body according to an embodiment of the present invention is schematically shown.

[0023] The above-mentioned drawings are part of the specification of the embodiments of the present invention, which illustrate exemplary embodiments of the present invention. The attached drawings and the description of the specification are used together to illustrate the principles of the embodiments of the present invention. It should be understood that the above general description of the drawings and the following specific implementations are only exemplary and illustrative, and they cannot limit the scope of the present invention. DETAILED DESCRIPTION

[0024] In order to make the objectives, technical solutions and advantages of the embodiments of the present invention more clearly understood, the spirit of the contents disclosed by the present invention will be clearly explained with the accompanying drawings and detailed descriptions below. After understanding the embodiments of the contents of the present invention, any technician in the relevant technical field can change and modify the techniques taught by the contents of the present invention without departing from the spirit and scope of the contents of the present invention.

[0025] The exemplary embodiments and descriptions of the present invention are used to explain the present invention, but are not intended to limit the present invention. In addition, elements / components with the same or similar reference numerals used in the drawings and embodiments are used to represent the same or similar parts.

[0026] The terms “first”, “second”, etc. used in the present invention do not particularly refer to an order or sequence, nor are they used to limit the present invention. They are only used to distinguish elements or operations described with the same technical terms.

[0027] The directional terms used in the present invention, such as up, down, left, right, front or back, etc., are only used to refer to the directions of the drawings. Therefore, the directional terms used are used to illustrate and not to limit the present invention.

[0028] The words “include,” “including,” “have,” “contain,” etc. used in the present invention are open-ended terms, meaning including but not limited to.

[0029] The term "and / or" used in the present invention includes any or all combinations of the items mentioned.

[0030] Regarding the present invention, "plurality" includes "two" and "more than two"; regarding the present invention, "plurality of groups" includes "two groups" and "more than two groups".

[0031] The terms "substantially" and "approximately" used in the present invention are used to modify any quantity or error that may vary slightly, but these slight changes or errors do not change their essence. Generally speaking, the range of slight changes or errors modified by such terms may be 20% in some embodiments, 10% in some embodiments, 5% in some embodiments, or other values. Those skilled in the art should understand that the aforementioned values ​​can be adjusted according to actual needs and are not limited thereto.

[0032] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0033] In the case of using expressions such as "at least one of A, B, and C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but not be limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or a system having A, B, C, etc.). In the case of using expressions such as "at least one of A, B, or C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, or C" should include but not be limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or a system having A, B, C, etc.). Those skilled in the art should also understand that any transitional conjunctions and / or phrases that substantially represent two or more optional items, whether in the specification, claims, or drawings, should be understood to give the possibility of including one of these items, either of these items, or both of these items. For example, the phrase "A or B" should be understood to include the possibilities of "A" or "B", or "A and B".

[0034] As mentioned in the background technology, the scheme of using visual-tactile fusion to realize intelligent body positioning and execution operation in the prior art usually uses cameras for high-precision positioning, which often requires very expensive depth cameras or laser radars, and the pre-calibration installation steps before use are relatively complicated and cumbersome. Moreover, it is necessary to frequently set up operations for different objects and different venues, which is time-consuming and labor-intensive, resulting in high costs and complex operations for this method. In addition, the existing methods of using visual and tactile information for positioning and operation processing are mostly based on neural networks for end-to-end processing, which often require a large number of data sets for each scene and each task, and the models trained with the data sets of each specific scene and specific object have poor generalization ability and can often only process the same tasks, resulting in a larger amount of data required and poor versatility. Moreover, the existing tactile sensors often need to use a specific neural network for filtering processing to obtain tactile feedback information with high availability, but this method will also further result in a large amount of data required and poor versatility. For example, when changing to a different tactile sensor, it needs to be retrained before it can be used, making its data processing method not universal.

[0035] For example, there is also a robot operation posture control method based on visual-tactile multi-scale positioning in the prior art. This method is mainly based on three stages: rough positioning of the operation target object by visual sensors, precise positioning of the operation target object by tactile sensors, and calculation of the robot operation posture. First, the operation environment and the state of the target object to be operated are measured by visual sensors. According to the collected image information and considering the safety distance, the estimated posture of the target object to be operated and the estimated target posture of the robotic arm system are preliminarily obtained; then, the relative posture deviation between the end of the robotic arm system and the target object to be operated is measured by tactile sensors; finally, according to the robot operation accuracy requirements, the estimated target posture of the robotic arm system is iteratively controlled and adjusted. The main purpose of this existing scheme is to accurately control the robot operation posture through the visual-tactile multi-scale positioning method, make up for the problems of light influence, background clutter interference, overlap and occlusion between objects in single visual measurement, increase local information by tactile perception, and improve the posture estimation accuracy of the target object to be operated. Among them, the existing technology still uses neural network modeling to complete the processing of displacement deviation and angle deviation to achieve precise positioning of the target object. It is mainly used to improve the adaptability of the intelligent agent to complex environments and background interference as well as object overlapping and occlusion during positioning execution, and cannot achieve the technical effect of reducing the cost of intelligent agent positioning execution and improving data versatility.

[0036] In view of at least one of the above problems, the embodiments of the present invention aim to provide a positioning execution control method, device, equipment, medium and product for an intelligent body with high precision, which has more universal data processing and simpler operation on the basis of lower cost, thereby providing a method that can achieve high-precision measurement of the six-dimensional information of the target object in space by using low-cost detection equipment such as tactile sensors and detection cameras, and can make full use of visual and tactile perception feedback to improve positioning efficiency and operational performance, while being able to achieve rapid adaptation across different intelligent body systems, thereby maintaining a higher level of intelligence.

[0037] One aspect of an embodiment of the present invention provides a positioning execution control method for an intelligent agent, which includes: determining the tactile detection range of a detection end of the intelligent agent based on a received initial visual image; controlling the detection end to perform touch detection within the tactile detection range to generate a contact detection image; and correcting the touch detection process of the detection end through the contact detection image to generate a contact positioning image, and the contact positioning image is used to control the execution end of the intelligent agent to perform positioning execution.

[0038] Figure 1 The application scenario diagram of the positioning execution control method, device, equipment, medium and program product of an intelligent body according to an embodiment of the present invention is schematically shown.

[0039] like Figure 1As shown, the application scenario 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0040] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only examples).

[0041] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0042] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0043] It should be noted that the positioning execution control method of the intelligent body provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the positioning execution control device of the intelligent body provided in the embodiment of the present invention can generally be set in the server 105. The positioning execution control method of the intelligent body provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Correspondingly, the positioning execution control device of the intelligent body provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105.

[0044] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0045] The following will be based on Figure 1 The scene described by Figure 2~Figure 4B The positioning execution control method of the intelligent agent of the disclosed embodiment is described in detail.

[0046] Figure 2 The flowchart of the positioning execution control method of an intelligent agent according to an embodiment of the present invention is schematically shown.

[0047] like Figure 2 As shown, one aspect of an embodiment of the present invention provides a positioning execution control method of an intelligent body, which includes operations S201 to S203.

[0048] In operation S201, a tactile detection range of a detection end of an agent is determined according to a received initial visual image; In operation S202, the detection end is controlled to perform touch detection within a tactile detection range to generate a contact detection image; and In operation S203, the contact detection image is used to correct the touch detection process of the detection end to generate a contact positioning image, and the contact positioning image is used to control the execution end of the intelligent agent to perform positioning execution.

[0049] The intelligent agent can be the executor of the above-mentioned positioning execution control method of the embodiment of the present invention, or it can be an executor controlled by the positioning execution control method. Specifically, it can be a humanoid intelligent robot or other artificial intelligence device, which usually has an actuator to complete specific action tasks. For example, a humanoid robot can use a mechanical dexterous hand to complete the action task of picking up objects, or even perform action tasks such as packaging products on an industrial production line. Specifically, embodied intelligence can perform operations similar to human unlocking, accurately locate the position of the lock, and perform actions such as accurately inserting the key and turning it to unlock it.

[0050] In an embodiment of the present invention, the agent can respond to the generated action execution control instruction to execute the target action task. The target action task can be provided to the agent by the user using the data input terminal of the agent (such as a microphone, a remote control, and a touch screen device) according to the actual scene requirements, or it can be an operation task that the agent creates autonomously according to its own work scene requirements, changes in the actual scene, or action needs. According to the execution task, the agent can generate the above-mentioned action execution control instruction to control the actuator to complete the execution task.

[0051] The initial visual image may be a computer vision image obtained by the agent by detecting the current environment information of the current space (such as indoors or outdoors) according to the execution requirements of the target task, and the acquisition method may be through a visual sensor (such as a simple camera). The current environment information may include the type, shape, size, position, quantity, color and other attribute information of objects, people and other agents in the current space, and may also include information such as the brightness of the environment. For example, when the home humanoid robot follows the user home and arrives at the door of the house, it can receive the user's "open the door" voice command through the microphone, and create an execution task of "unlock and open the door" in response to the voice command, and detect the object information such as the position (such as the position of the door and the position of the door lock), type, size and other object information in the current environment, as well as the number, appearance, body shape and action behavior of the surrounding people. In addition, the brightness and visibility of the current door can also be detected.

[0052] The detection end of the intelligent body can be an actuator of an intelligent body with contact perception, such as a mechanical dexterous hand and arm structure with simulated skin, etc., without specific limitation.

[0053] The tactile detection range can be the object area range of the agent's contact perception based on the target action task, and is the operation area where the operation position of the positioning execution operation is located. For example, for the target action task of "unlocking and opening the door", the tactile detection range can be the area occupied by the door lock in the door area. Through image analysis of the initial visual image, the agent can distinguish between the objects and people "observed", and preliminarily realize the recognition of the area where the execution position of the target action task is located. The area where the execution position of the target animal task is located can be understood as the above-mentioned tactile detection range. Therefore, through the acquisition and recognition of the initial visual image, the approximate spatial area and range of the positioning execution position can be quickly obtained.

[0054] In an embodiment of the present invention, the detection end structure may have a visual touch sensing structure composed of a visual sensor (such as a camera) and a relatively designed optical path structure. Of course, it may also include a corresponding piezoelectric sensor to satisfy the pressure touch, and the specific adjustment may be made according to actual needs. For example, the optical path structure may be an annular light strip composed of a plurality of LED lights, and at least one camera is arranged on the inner side of the optical path structure as a visual sensor to obtain an optical path change image of the optical path structure. In addition, a layer of silicone or rubber layer may be laid on the outer side of the optical path structure as a contact layer, wherein the contact layer may further design some elastic recovery structures based on the elastic recovery of the silicone or rubber layer to simulate the rebound effect of real skin after contact separation. When the contact surface of the contact layer contacts the surface of the object, the contact layer is deformed, causing the optical path structure of the annular light strip to undergo a physical change, thereby generating an optical path change, and the camera may detect the optical path change in time to generate a contact detection image.

[0055] The detection end that contacts the surface of the object may have a contact surface, which is formed by the outer surface of the contact layer. During the touch detection process, the center position of the contact surface needs to fall within the above-mentioned tactile detection range.

[0056] Therefore, when the detection end performs touch detection within the tactile detection range, the detection image of a certain position in the tactile detection range of the contact layer can be obtained through the above-mentioned visual touch sensing structure, wherein the contact layer changes due to the contact force, driving the optical path structure to produce an optical path change, and this optical path change enables the visual sensor in the detection state to generate a corresponding detection image, that is, a contact detection image. In other words, the above-mentioned method of the embodiment of the present invention can visualize the tactile information, so that the optical path structure and the internal perception sensor can be protected by the contact layer, and at the same time, the tactile sense of the visual touch sensor is less likely to be disturbed by the external environment, thereby ensuring the stability of the perception process and having better robustness.

[0057] It can be seen that the touch sense is visualized by means of the contact detection image, which ensures the accurate positioning of the contact position between the detection end and the object surface corresponding to the tactile detection range under the condition of contact perception. Therefore, if the contact detection image generated by the current touch detection cannot be determined as the final operation position for positioning execution, the touch detection position can be further corrected, and the touch detection is performed again on the object surface within the tactile detection range to generate a new contact detection image, so as to further determine whether the contact position corresponding to the contact detection image is the operation position for positioning execution.

[0058] Among them, although the initial visual image can provide the tactile detection range of the detection end and roughly determine the operation range of the positioning execution, it still cannot accurately locate the execution position. Further combined with the touch detection of the detection end in the tactile detection range, the position of the tactile detection range can be located and identified to ensure that the accurate positioning execution operation position can be obtained in the shortest time. For example, in the above-mentioned target action task of "unlocking and opening the door", the intelligent agent can determine the spatial area of ​​the door and the spatial area of ​​the door lock on the door (i.e., the tactile detection range) through the initial visual image, and then repeatedly touch the spatial area of ​​the door lock through the specific fingers of the dexterous hand. If each touch cannot locate the final positioning execution operation position, the touch position and detection posture are corrected, and touch other positions are made within the tactile detection range. Finally, the final contact detection image can be obtained as a contact positioning image through the visual touch sensor. The contact positioning image can be the imaging of the surface of the object to be executed by the target action task formed by the positioning execution operation position. Therefore, the data generated by the continuous analysis of the contact detection image can be used as a basis for the correction of the touch position and detection posture of the detection end during the touch detection process.

[0059] In summary, according to the above positioning execution control method of the embodiment of the present invention, by generating a contact detection image and correcting the touch detection process of the detection end according to the contact detection image, it is possible to obtain the tactile detection range by obtaining the initial visual image through a low-cost camera to roughly estimate the object's posture, and then continuously correct and adjust the touch detection within the tactile detection range in conjunction with the visual-tactile sensing structure of the detection end, so that the object and the visual-tactile sensing structure are fully in contact and form a detection image, that is, the precise calculation of the operation position of the positioning execution can be achieved, so that the execution end can be controlled to perform the task at the operation position. For example, the keyhole can be the execution position of the intelligent agent, and the intelligent agent can control the key to be inserted into the keyhole and perform the rotation operation to achieve the target action task of "opening the door".

[0060] Among them, the execution end of the intelligent agent can be another actuator that exists relative to the detection end, or it can be the detection end itself (that is, the detection end also serves as the execution end). For example, the left hand of the intelligent agent can serve as the detection end, the right hand can serve as the execution end, or the same hand can serve as both the detection end and the execution end. There is no specific limitation.

[0061] In summary, compared with the traditional solution of realizing positioning execution by visual-touch fusion based on neural network calculation, the above-mentioned positioning execution control method of the embodiment of the present invention can only use lower-cost visual and tactile perception devices (visual-touch perception structure) to achieve high-precision positioning execution operations, wherein the installation and calibration of these perception devices and other operations are simpler and more convenient, and there is no need to perform frequent setting operations for different objects and different scenes, saving time and effort. Moreover, the processing of positioning execution operations based on visual information and tactile information no longer needs to strictly rely on the end-to-end processing method based on neural networks, so that it can achieve higher versatility for different scenes and different execution tasks, and no longer requires higher data volume in the traditional method. Therefore, even if low-cost tactile sensors and detection cameras and other detection equipment are used, high-precision measurement of the six-dimensional information of the target object in space can be achieved, and visual and tactile perception feedback can be fully utilized to improve positioning efficiency and operating performance, while being able to achieve rapid adaptation across different intelligent systems, with higher stability and versatility, a wider range of applications, and a higher level of intelligence.

[0062] Figure 3 A flowchart of an application scenario of a positioning execution control method of an intelligent body according to an embodiment of the present invention is schematically shown.

[0063] like Figure 2 and Figure 3 As shown, according to one embodiment of the present invention, before operation S201 determines the tactile detection range of the detection end of the intelligent agent according to the received initial visual image, it also includes: Perform visual detection on the target detection space where the agent is located to generate an initial visual image.

[0064] like Figure 3 In the operation 310 shown, the target detection space can be the space where the agent is located when performing visual detection for positioning and executing the target action task. For example, if the target action task of the agent is "unlocking and opening the door", if the agent is standing in the corridor outside the door, the space of the corridor can be used as the target detection space. Visual detection can be understood as the "viewing" process of the agent, that is, the process of the agent using its own visual sensor (such as a simple low-cost RGB camera) to perform image detection on the surrounding environment or target objects.

[0065] In response to the target action task, the agent can obtain the corresponding initial visual image I at the current moment by "observing" the target detection space. camera, such as the detection image of the target object in front of the agent in the space where the agent is currently located. For example, for the target action task of "unlocking and opening the door", the initial visual image may include the door in front of the agent, the wall around the door, and part of the extended environment of the corridor, and may also include some humans, objects, and even pets.

[0066] Therefore, even very cheap visual sensors can easily acquire the initial visual image, which greatly reduces the image acquisition cost compared to the existing visual perception technology that usually uses high costs and complex settings. It also reduces the complexity of parameter settings and greatly simplifies the detection method.

[0067] like Figure 2 and Figure 3 As shown, according to one embodiment of the present invention, in operation S201, determining the tactile detection range of the detection end of the intelligent agent according to the received initial visual image includes: performing object position parsing on the initial visual image according to the target execution object; The tactile detection range is determined based on the object position resolution.

[0068] The target execution object can be the target object that the agent operates to complete the positioning execution. For example, in the case where the aforementioned target action task is "unlock and open the door", the lock can be used as the target execution object. The agent itself has the corresponding image parsing and processing capabilities, and can segment and identify objects in the initial visual image and identify the target objects in the image. At the same time, the agent can also obtain the position information of each object identified from the initial visual image in the target detection space coordinate system based on the "observation" data (such as spatial coordinate data) of the target detection space (and can also identify its type, size, shape and other related information), that is, perform object position parsing.

[0069] Among them, for the recognition of the target execution object in the initial visual image, the intelligent agent can perform feature comparison with the object image segmented and extracted from the initial visual image based on the image data related to the target execution object crawled from the network in real time or stored historically in the historical database, so as to identify the object with the closest features to the target execution object and obtain the spatial position of the object in the target detection space, which will not be elaborated here.

[0070] Furthermore, since the target execution object can be obtained by analyzing and extracting the initial visual image, the position information of the target execution object in the target detection space can also be determined. Based on this position information, the position and size of the spatial area occupied by the target execution object in the target detection space can be determined, that is, the tactile detection range is obtained.

[0071] Specifically, the position coordinates of each pixel point of the target execution object mapped to the space in the initial visual image can be obtained. Due to the use of a camera with poor performance, it is possible that light interference phenomena such as shadows produced by the target execution object will also be generally regarded by the intelligent agent as being within the area of ​​the tactile detection range.

[0072] Therefore, through the above-mentioned object position analysis for the initial visual image, a relatively simple object detection technology can be used, that is, the tactile detection range can be roughly determined directly, quickly and effectively.

[0073] like Figure 2 and Figure 3 As shown, according to an embodiment of the present invention, in operation S202, controlling the detection end to perform touch detection within the tactile detection range to generate a contact detection image includes: Obtaining the touch detection execution position within the touch detection range; The detection end is controlled to perform touch detection on the touch detection execution position to generate a contact detection image.

[0074] like Figure 3 In the operation S320 shown, after determining the tactile detection range in which the intelligent detection terminal can perform touch detection, it is also necessary to consider the touch detection position (i.e., touch detection execution position) of the object surface within the area defined by the tactile detection range each time the detection terminal performs touch detection, and the touch detection execution position must fall within the tactile detection range. Those skilled in the art should understand that if the touch detection execution position is a contact surface, at least the center point of the contact surface must fall within the tactile detection range.

[0075] Typically, the agent can roughly determine a contact position that is most likely to be close to the positioning execution position as the starting touch detection position of the touch detection execution based on the comparison result of the overall shape of the tactile detection range and the overall shape of the target execution object.

[0076] For example, in the case where the target action task is "unlock and open the door", the lock as the target execution object can be a spherical object. The intelligent agent can roughly determine an image area that is approximately elliptical or circular as the tactile detection range based on the general position of the lock on the door (such as the lock is generally located in the lower middle of the door, close to the edge of the door frame and below the door handle) through the recognition and extraction of the initial visual image. Furthermore, the keyhole can be used as the operation position for the positioning execution of the target execution object. Therefore, the intelligent agent can also further determine the approximate position of the keyhole in the limited area of ​​the tactile detection range based on the central position of the keyhole in the spherical lock, and this approximate position can be used as the starting touch position corresponding to the touch execution position.

[0077] For the touch detection process after data correction, the touch detection execution position is the touch detection position generated after data correction is performed on the previous touch detection execution position, and the touch detection position will also fall within the tactile detection range at the same time.

[0078] According to the touch detection information such as the above-mentioned tactile detection range and the corresponding touch detection execution position, the controller of the detection end (such as Exploration Arm Controller) can further perform touch detection on the touch detection execution position located in the tactile detection range according to the current execution state of the detection end (such as Exploration Arm). The current execution state of the detection end may include the name of the action currently executed by the detection end, motion parameters (such as the rotation angle of each finger joint, etc.), spatial position, angle of the contact surface, and other state information related to the posture of the detection end.

[0079] Therefore, according to the current execution state of the detection end, the detection end can accurately calculate the required posture adjustment data to meet the touch detection requirements of the touch detection execution position, thereby generating a corresponding contact detection image (ie, Contact Image) according to the visual touch sensing structure of the touch detection contact surface. tactile .

[0080] The contact detection image may be an image obtained by the detection end performing touch detection based on continuous posture and position correction, and only the last contact detection image obtained by the detection end without posture and position correction can be used as the contact positioning image. Therefore, the contact detection image can be the basis for the above detection end to continuously perform touch detection correction.

[0081] Therefore, by determining the touch detection execution position, a touch detection closer to the positioning execution position can be determined intelligently, thereby greatly reducing the number of corrections to the touch detection process and reducing the correction amplitude, so that the detection resources of the detection end are occupied less, and the contact positioning image can be accurately obtained in the shortest time. Furthermore, the touch detection correction is performed according to the current execution state of the detection end, so that each touch detection can have a more accurate and stable contact detection image.

[0082] like Figure 2 and Figure 3 As shown, according to an embodiment of the present invention, in operation S203, the contact detection image is used to correct the touch detection process of the detection end to generate the contact positioning image, including: Acquire plane position correction data and attitude angle correction data through contact detection images; updating the touch detection execution position of the detection end within the tactile detection range according to the plane position correction data; and adjusting the touch detection angle of the detection end when performing touch detection at the updated touch detection execution position according to the posture angle correction data; The detection end is controlled to perform touch detection on the updated touch detection execution position according to the touch detection angle until a contact positioning image is generated.

[0083] exist Figure 3 In the operations S330-S331 and S332 shown, the touch position and touch posture of the next touch detection of the detection end can be corrected according to the analysis of the contact detection image. Among them, the visual touch image of the target execution object with the target execution position can be obtained in advance according to the model such as Model Mesh by using simulation (such as Taxim Simulation), and the visual touch image can be used as the detection comparison image (Target Image) of the target execution object.

[0084] Compared with directly filtering the contact detection image through a neural network, in an embodiment of the present invention, differential amplification processing can be performed on the contact detection image and the above-mentioned detection comparison image to generate a detection processing image. Since the foreground and background of the detection processing image are significantly different and the sensitivity can be adjusted, it has higher versatility and strong migration ability, and also avoids the situation where the contact detection image after neural network filtering needs to re-collect a large amount of data for each sensor for re-training.

[0085] Considering that in practical applications, the initial visual image can only provide an approximate position for the positioning execution of the target execution object, and when it is applied to different detection ends of different intelligent bodies, it will also bring huge calculation errors to the theoretical data due to differences in the structural design of the visual-tactile sensing structure, differences in the softness and hardness of the contact surface, etc., which will seriously affect the accurate judgment of the touch position, increase the number of touch processes, and extend the touch time.

[0086] To this end, it is necessary to consider that the surface area of ​​the object in the tactile detection range of the target execution object and the visual touch sensing structure are in full contact during the touch detection process to avoid the consequences of the above errors. The full contact of the touch detection needs to meet the following two conditions: the first is that the part of the area corresponding to the tactile detection range of the object surface exceeds the measurement range of the visual touch sensing structure. At this time, it is necessary to correct the touch position of the visual touch sensing structure of the detection end in the plane direction to make it possible for the two to be in full contact; the second is that the surface of the object corresponding to the tactile detection range and the contact surface of the visual touch sensing structure are in contact but there is an inclined angle. At this time, it is necessary to adjust the posture of the detection end so that the inclined angle formed between the contact surfaces of the visual touch sensing structure is 0, so that the two can be in full contact.

[0087] By performing image analysis on the detection processing image generated after the above-mentioned contact detection image processing and the above-mentioned detection comparison image (Target Image), and comparing and estimating the image analysis data of the two, the detection end plane position deviation and posture angle deviation corresponding to the touch detection position in the acquisition process of the current contact detection image are obtained. For example, as the comparison object of the contact detection image, the detection comparison image can be a simulation image that is closest to the contact positioning image. By performing image analysis on it, the standard position of each pixel point can be obtained. Therefore, the difference between the detection position of each pixel point obtained by the image analysis of the contact detection image and the standard position can be used to determine whether the contact detection image has a missing pattern (such as part of the contact area is not in the tactile detection range) and an offset (such as the center of the contact area is offset), whether the pattern in the image is deformed (such as the uneven contact surface causes an angle), etc. In this way, the corresponding plane position correction data (i.e., Plane Correction) (Δx, Δy) and posture angle correction data (i.e., Orientation) R (θ, v) can be obtained very accurately.

[0088] When the value of the above plane position correction data (Δx, Δy) exceeds a certain threshold, it can be determined that position deviation has occurred. Figure 3 Operation S332 is shown, sending the above-mentioned plane position correction data (Δx, Δy) to the detection end controller (such as Exploration Arm Controller), so that the detection end controller can correct the execution position of the next touch detection execution according to the current execution state of the detection end at the current moment. The corrected position can be the touch detection position to be executed by the detection end in the next touch detection execution process.

[0089] Accordingly, when the value of the above-mentioned attitude angle correction data R (θ, v) exceeds a certain threshold, it can be determined that an angle deviation has occurred. Figure 3 Operation S331 is shown, sending the above-mentioned posture angle correction data R (θ, v) to the detection end controller, so that the detection end controller can correct the detection end touch angle at the next touch detection execution according to the current execution state of the detection end at the current moment, and specifically can be an angle adjustment of the contact surface of the visual touch sensor structure. The corrected angle can be the touch angle of the contact surface of the visual touch sensor to be executed by the detection end when executing the next touch detection execution process.

[0090] Among them, the above-mentioned plane position correction and attitude angle correction can be carried out simultaneously. The detection end will update the execution position of the next touch measurement according to the plane position correction data to obtain a new touch measurement execution position. At the same time, it will also correct the inclination angle of the contact surface of the visual touch tactile structure when performing the next touch measurement at the new touch measurement execution position according to the attitude angle correction data, thereby ensuring that the next touch measurement is strictly performed in accordance with the new touch measurement execution position and the inclination angle of the new contact surface.

[0091] Of course, the plane position correction and attitude angle correction can also be performed separately. This situation usually occurs when one of the aspects does not need to be corrected. For example, if there is a plane position deviation but no angle deviation, then only the plane position needs to be corrected.

[0092] Therefore, the detection end only needs to perform the next touch detection according to the corrected touch detection angle and the corresponding touch detection execution position, and generate a new contact detection image. Further, according to the above-mentioned deviation judgment and deviation correction for the new contact detection image, another touch detection is performed again according to the latest correction data... until a contact detection image without any deviation can be generated as a contact positioning image. In this process, the detection end also needs to continuously adjust the required contact force according to the need for deviation correction, so as to ensure a more sufficient contact effect with the surface of the touch detection execution position based on the deviation correction.

[0093] Therefore, by analyzing and judging the contact detection image each time, the next touch detection process can be corrected according to the judgment result, so as to achieve repeated correction of the touch detection process performed by the detection end, so that the contact surface of the visual touch sensor of the detection end continuously approaches the target area on the target execution object, and generates the final contact positioning image. At this time, the touch detection correction process can be terminated.

[0094] Therefore, with the help of the above-mentioned repeated corrections to the touch detection process, even if the traditional calculation method based on large-scale data neural network models is not adopted, high-precision positioning detection can be obtained through lower-cost perception technology, effectively avoiding the traditional end-to-end processing solution, and being able to implement the positioning detection process based on different intelligent agents for different scenarios and different tasks, achieving higher versatility, while also avoiding more data volume, and no longer requiring frequent parameter settings and data changes.

[0095] Figure 4A The application scenario diagram of the plane position correction of the positioning execution control method of the intelligent body according to the embodiment of the present invention is schematically shown.

[0096] like Figure 2 and Figure 3As shown, according to one embodiment of the present invention, before obtaining the plane position correction data and the posture angle correction data through the contact detection image, it also includes: A target imaging region of a target execution object in the contact detection image is identified.

[0097] like Figure 4A As shown in FIG. 1 , before obtaining the plane position correction data, the contact detection image needs to be analyzed to obtain the area in the contact detection image that reflects the target operation position. The area is the area where the target object performs the positioning operation, that is, the target imaging area. By analyzing and extracting features from the image, the position of the target imaging area in the contact detection image can be determined, as shown in FIG. Figure 4A The concentric circle area enclosed by the blue box is shown (Circle 0.97). The target imaging area can be identified by using a YOLO model trained with a small amount of data.

[0098] It should be noted that, since the visual touch sensing structure has an optical path structure (such as a ring-shaped LED light strip), the imaging of the contact detection image within the tactile detection range of the target execution object is not consistent with the actual object surface of the target execution object, but the contact detection image is generated based on the image generated by the optical path change of the optical path structure, for example Figure 4A Colored concentric ring image shown.

[0099] Therefore, the target imaging area can be quickly identified in an extremely simple manner, thereby determining the imaging position of the target execution object in the contact detection image, so that its deviation correction data can be determined more quickly subsequently, thereby ensuring the accuracy of its deviation correction data.

[0100] like Figure 2-4A As shown, according to one embodiment of the present invention, in acquiring plane position correction data through contact detection images, the method includes: The target imaging area and the imaging field of view area of ​​the contact detection image are compared to obtain plane position correction data.

[0101] like Figure 4A As shown, by identifying the target imaging area, the position of the target execution object in the contact detection image formed by the visual touch sensing structure can be determined. Among them, the imaging range of the contact detection image can be determined according to the imaging field of view of the visual touch sensing structure, such as Figure 4A The largest black background area shown can be regarded as the imaging area of ​​the imaging field of view of the visual touch sensing structure, that is, the imaging field of view area. Figure 4A As shown, the center of the target imaging area is B, and the center of the imaging field of view area is A. When the two do not overlap, it can be explained that there is a plane position deviation in the corresponding touch measurement execution.

[0102] Therefore, the plane position correction data can be expressed by the following formula (1):

[0103] Wherein, b refers to the reference coordinate system of the detection end, ee1 represents the end of the detection end, tac refers to the visual touch sensing structure, Represent the position coordinates of the center of the target imaging area and the center of the imaging field of view in the image, respectively, and Represents the distance of each pixel in reality. The upper left corner of T represents the reference coordinate system, the lower left corner represents the target coordinate system, and T represents the conversion relationship from the reference coordinate system to the target coordinate system.

[0104] Among them, the position of the center B of the target imaging area in the imaging field of view area can also represent the plane position correction data required in the next touch correction process. Moreover, the calculation result of the area size of the target imaging area in the imaging field of view area can determine whether the target imaging area exceeds the imaging field of view, in other words, whether the corresponding detection end at least partially exceeds the boundary of the tactile detection range or exceeds the boundary of the imaging field of view during touch detection. Therefore, in the next touch execution process after the corresponding correction, under the imaging visual angle, the detection end should move the center A of the detection imaging field of view obliquely toward the target imaging area B to complete the position correction. The correction of the above position is essentially to make the center B of the target imaging area coincide with the center A of the imaging field of view area in the contact detection image generated by the next touch execution.

[0105] Therefore, through a very simple technical principle, the accurate acquisition of planar position correction data can be quickly achieved, and high-precision measurement of the planar information of the target object in space can be achieved, thereby ensuring positioning efficiency and subsequent operational performance. It can achieve rapid adaptation across different intelligent systems and has higher versatility.

[0106] Figure 4B The application scenario diagram of the posture angle correction of the positioning execution control method of the intelligent body according to the embodiment of the present invention is schematically shown.

[0107] like Figure 2-4B As shown, according to one embodiment of the present invention, in acquiring posture angle correction data through contact detection images, it includes: extracting a filtered contact image of the contact detection image; The correspondence between each pixel point on the filtered contact image and the preset simulated contact image is obtained to obtain the posture angle correction data.

[0108] As Figure 4BAs shown, after filtering the contact detection image with a custom filter (Custom Filter), a filtered contact image can be generated. The filtered contact image is further subjected to image analysis processing to extract pixel information of each imaged pixel point in the filtered contact image, and the pixel information may include position information of the corresponding pixel point. Among them, the graphics in the filtered contact image may be incompletely imaged and may be distorted due to the angle between the contact surface of the detection end and the inability to fully contact, such as Figure 4B The pattern shown shows only the upper left corner of a partial ring.

[0109] The preset simulated contact image may be the aforementioned detection contrast image (Target Image). Similar to the image analysis process of the above-mentioned filtered contact image, pixel information of each imaging pixel point of the preset simulated contact image may be extracted.

[0110] Furthermore, the pixel information of the pixel points of the filtered contact image is compared with the pixel information of the pixel points of the preset simulated contact image and a corresponding relationship between the pixel points is established. For example, the pixel information of the preset simulated contact image can be used as a reference standard to calculate the comparison score of each pixel point of the filtered contact image, and then the positional relationship between the filtered contact image and the preset simulated contact image can be determined, that is, the angle difference between the filtered contact image and the actual imaging requirement to be achieved can be obtained, and then the above-mentioned posture angle correction data can be obtained to determine the angle at which the detection end of the next touch detection execution process needs to adjust the contact surface of the visual touch sensing structure according to the current execution state.

[0111] The attitude angle correction data satisfies the following formulas (2)-(4):

[0112] Where β is the inclination angle of rotation: like Figure 4B As shown, the figure circled by the white box at the bottom right can be the imaging of the target execution object analyzed according to the filtered contact image. At this time, if the angle correction is to be performed, it is necessary to rotate the figure to the other side according to a virtual rotation axis to complete the imaging. The actual rotation angle required is the correction value of the posture angle.

[0113] Therefore, the attitude angle correction data can be obtained quickly and accurately, and the deviation angle information of the target object in space can be measured with high precision, thereby ensuring the positioning efficiency and subsequent operational performance. It can achieve rapid adaptation across different intelligent systems and has higher versatility. In this process, large amounts of data training and processing are no longer required, which is more cost-effective and can ensure the accuracy of the data without any negative impact on the intelligence level of the intelligent body.

[0114] In summary, the positioning execution control method of an intelligent body provided by an embodiment of the present invention can at least partially solve at least one of the technical problems existing in the related art, and thus can achieve at least one of the following technical effects: First of all, only lower-cost visual and tactile sensing devices can be used to achieve high-precision positioning and execution operations. The installation and calibration of these sensing devices are simpler and more convenient, and there is no need for frequent setup operations for different objects and scenes, saving time and effort.

[0115] Secondly, the processing of positioning and executing operations based on visual and tactile information no longer needs to strictly rely on end-to-end processing based on neural networks, which makes it possible to achieve higher versatility for different scenarios and different execution tasks, and no longer requires higher data volumes in traditional methods.

[0116] Therefore, even low-cost tactile sensors, detection cameras and other detection equipment can be used to achieve high-precision measurement of the six-dimensional information of the target object in space, and visual and tactile perception feedback can be fully utilized to improve positioning efficiency and operational performance. At the same time, it can achieve rapid adaptation across different intelligent body systems, has higher versatility, a wider range of applications, and can maintain a higher level of intelligence.

[0117] Based on the above-mentioned intelligent body positioning execution control method, the present invention also provides an intelligent body positioning execution control device. Figure 5 The device is described in detail.

[0118] Figure 5 The structure block diagram of the positioning execution control device of an intelligent body according to an embodiment of the present invention is schematically shown.

[0119] like Figure 5 As shown, the positioning execution control device 500 of the intelligent body in this embodiment includes a range determination module 510, an image generation module 520 and a touch correction module 530.

[0120] The range determination module 510 is used to determine the tactile detection range of the detection end of the agent according to the received initial visual image. In one embodiment, the range determination module 510 can be used to perform the operation S201 described above, which will not be repeated here.

[0121] The image generation module 520 is used to control the detection end to perform touch detection within the tactile detection range to generate a contact detection image. In one embodiment, the image generation module 520 can be used to perform the operation S202 described above, which will not be repeated here.

[0122] The touch correction module 530 is used to correct the touch process of the detection end through the contact detection image to generate a contact positioning image, and the contact positioning image is used to control the execution end of the intelligent agent to perform positioning execution. In one embodiment, the touch correction module 530 can be used to perform the operation S203 described above, which will not be repeated here.

[0123] According to an embodiment of the present invention, any multiple modules of the range determination module 510, the image generation module 520 and the touch correction module 530 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the range determination module 510, the image generation module 520 and the touch correction module 530 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the range determination module 510, the image generation module 520 and the touch correction module 530 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0124] Figure 6 A block diagram of an electronic device suitable for implementing a positioning execution control method for an intelligent body according to an embodiment of the present invention is schematically shown.

[0125] The electronic device provided by an embodiment of the present invention includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the positioning execution control method of the above-mentioned intelligent body.

[0126] like Figure 6As shown, the electronic device 600 according to an embodiment of the present invention includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 to a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include an onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0127] In RAM 603, various programs and data required for the operation of electronic device 600 are stored. Processor 601, ROM 602 and RAM 603 are connected to each other via bus 604. Processor 601 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 602 and / or RAM 603. It should be noted that the program can also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in the one or more memories.

[0128] According to an embodiment of the present invention, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to the bus 604. The electronic device 600 may further include one or more of the following components connected to the I / O interface 605: an input portion 606 including a keyboard, a mouse, etc.; an output portion 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 608 including a hard disk, etc.; and a communication portion 609 including a network interface card such as a LAN card, a modem, etc. The communication portion 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage portion 608 as needed.

[0129] The present invention also provides a computer-readable storage medium on which executable instructions are stored. When the instructions are executed by a processor, the processor executes the positioning execution control method of the above-mentioned intelligent body.

[0130] The computer-readable storage medium may be included in the device / apparatus / system described in the above embodiment; or it may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0131] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 602 and / or RAM 603 described above and / or one or more memories other than ROM 602 and RAM 603.

[0132] An embodiment of the present invention also includes a computer program product, which includes a computer program, and when the computer program is executed by a processor, the positioning execution control method of the above-mentioned intelligent body is implemented.

[0133] The computer program includes program codes for executing the method shown in the flow chart. When the computer program product is run in a computer system, the program codes are used to enable the computer system to implement the method provided by the embodiment of the present invention.

[0134] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when it is executed by the processor 601. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0135] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 609, and / or installed from a removable medium 611. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0136] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0137] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0138] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0139] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present invention may be combined and / or combined in various ways, even if such combinations and / or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments and / or claims of the present invention may be combined and / or combined in various ways without departing from the spirit and teachings of the present invention. All of these combinations and / or combinations fall within the scope of the present invention.

[0140] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination. The scope of the present invention is defined by the attached claims and their equivalents. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A positioning execution control method for an intelligent agent, characterized in that: include: Determining a tactile detection range of a detection end of the agent according to the received initial visual image; Controlling the detection end to perform touch detection within the tactile detection range to generate a contact detection image; as well as The contact detection image is used to correct the touch detection process of the detection end to generate a contact positioning image, and the contact positioning image is used to control the execution end of the intelligent body to perform positioning execution.

2. The method according to claim 1, characterized in that Before determining the tactile detection range of the detection end of the intelligent agent according to the received initial visual image, the method further includes: Perform visual detection on the target detection space where the intelligent agent is located to generate the initial visual image.

3. The method according to claim 1, characterized in that In determining the tactile detection range of the detection end of the intelligent agent according to the received initial visual image, the method includes: performing object position parsing on the initial visual image according to a target execution object; The tactile detection range is determined according to the object position analysis.

4. The method according to claim 1, characterized in that: In the step of controlling the detection end to perform touch detection within the tactile detection range to generate a contact detection image, the method includes: Acquiring a touch detection execution position within the tactile detection range; The detection end is controlled to perform touch detection on the touch detection execution position to generate a contact detection image.

5. The method according to claim 1, characterized in that In the correcting the touch detection process of the detection end by using the contact detection image to generate the contact positioning image, the method includes: Acquiring plane position correction data and attitude angle correction data through the contact detection image; updating the touch detection execution position of the detection end within the tactile detection range according to the plane position correction data; and adjusting the touch detection angle of the detection end when performing touch detection at the touch detection execution position after the update according to the posture angle correction data; The detection end is controlled to perform touch detection on the updated touch detection execution position according to the touch detection angle until the contact positioning image is generated.

6. The method according to claim 5, characterized in that Before acquiring the plane position correction data and the posture angle correction data through the contact detection image, the method further includes: A target imaging region of a target execution object in the contact detection image is identified.

7. The method according to claim 6, characterized in that In the step of acquiring the plane position correction data through the contact detection image, the method comprises: The target imaging area and the imaging field area of ​​the contact detection image are compared to obtain the plane position correction data.

8. The method according to claim 5, characterized in that In the step of acquiring posture angle correction data through the contact detection image, the method includes: extracting a filtered contact image of the contact detection image; The corresponding relationship between each pixel point on the filtered contact image and the preset simulated contact image is obtained to obtain the posture angle correction data.

9. A positioning execution control device for an intelligent body, characterized in that: include: A range determination module, configured to determine a tactile detection range of a detection end of the agent according to the received initial visual image; An image generation module, used for controlling the detection end to perform touch detection within the tactile detection range to generate a contact detection image; as well as The touch detection correction module is used to correct the touch detection process of the detection end through the contact detection image to generate a contact positioning image, and the contact positioning image is used to control the execution end of the intelligent body to perform positioning execution.

10. An electronic device comprising: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors execute the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images