A target recognition method and system
By acquiring the pose information of the center point connecting the eyes of the AR glasses wearer and the target depth information, coordinate transformation and matching are performed, solving the problem of deviation between virtual information and real targets in AR glasses and improving the user experience.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HUBEI XINGJI MEIZU TECH CO LTD
- Filing Date
- 2022-12-30
- Publication Date
- 2026-07-10
AI Technical Summary
The camera module on AR glasses has a pose difference with the human eye, which causes a discrepancy between the virtual information and the target actually seen by the human eye, resulting in a need to improve the user experience.
By acquiring the pose information of the center point of the line connecting the wearer's eyes in the smart wearable device, and combining it with the target's area range and depth information, coordinate system transformation and matching are performed to determine the target of interest and display the corresponding virtual information.
It improves the accuracy of AR glasses in recognizing targets of interest to users and enhances the user experience, achieving precise alignment between virtual information and real-world scenes.
Smart Images
Figure CN116206225B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of augmented reality technology, and in particular to a target recognition method and system. Background Technology
[0002] Augmented Reality (AR) is a technology that cleverly integrates virtual information with the real world. It simulates and applies computer-generated text, images, 3D models, music, videos, and other virtual information to the real world, thereby "enhancing" the real world.
[0003] As the primary application of AR technology, AR glasses suffer from a certain degree of deviation between the virtual information and the actual target seen by the human eye due to the positioning difference between the camera module and the human eye. This results in a need for further improvement in user experience. Summary of the Invention
[0004] In a first aspect, embodiments of this application provide a target recognition method applied to a smart wearable device, comprising:
[0005] Obtain the first pose information of the center point of the line connecting the eyes of the wearer of the smart wearable device;
[0006] Obtain the area range and depth information of multiple targets determined by the smart wearable device;
[0007] Using the depth information of each target, the preset information of the first position information in the first pose information is replaced to obtain multiple second position information, and the coordinate system is transformed for each second position information to obtain the corresponding pixel coordinate information;
[0008] The pixel coordinate information and the region range information are matched, and the target of interest is determined from among the multiple targets based on the matching result.
[0009] In some embodiments, the method further includes:
[0010] Obtain information about the target of interest and display the information about the target of interest in the display area corresponding to the target of interest.
[0011] In some embodiments, obtaining the area range information of multiple targets determined by the smart wearable device includes:
[0012] The regions of each target are marked using a target detection algorithm to obtain the corresponding target regions and region range information.
[0013] In some embodiments, obtaining depth information of multiple targets determined by the smart wearable device includes:
[0014] Based on each of the target regions, obtain the corresponding target recognition box;
[0015] The depth information corresponding to each target region is determined based on the proportional mapping relationship between the height pixel value and the depth information of each target recognition box. The proportional mapping relationship is constructed by the height pixel value of the sample target recognition box and the sample depth information corresponding to the sample target recognition box.
[0016] In some embodiments, obtaining depth information of multiple targets determined by the smart wearable device includes:
[0017] Based on the depth sensor, the depth information corresponding to each of the target regions is obtained.
[0018] In some embodiments, the step of using the depth information of each of the targets to replace the preset information of the first position information in the first pose information to obtain multiple second position information, and performing coordinate system transformation on each of the second position information to obtain the corresponding pixel coordinate information, includes:
[0019] Based on the depth information, obtain the average depth value corresponding to each target area;
[0020] By replacing the coordinate values corresponding to the Z-axis of the first position information in the first pose information with each of the average depth values, multiple second position information are obtained;
[0021] After transforming multiple second position information from the world coordinate system to the camera coordinate system of the smart wearable device, and then transforming them from the camera coordinate system of the smart wearable device to the pixel coordinate system, the corresponding pixel coordinate information is obtained.
[0022] In some embodiments, matching the pixel coordinate information and the region range information, and determining the target of interest among a plurality of targets based on the matching result, includes:
[0023] The pixel coordinate information is matched with the corresponding region range information, and the target corresponding to the region range information including the pixel coordinate information is taken as the target of interest.
[0024] Secondly, embodiments of this application also provide a target recognition system applied to a smart wearable device, comprising:
[0025] The first acquisition module is used to acquire the first pose information of the center point of the line connecting the eyes of the wearer of the smart wearable device.
[0026] The second acquisition module is used to acquire the area range information and depth information of multiple targets determined by the smart wearable device;
[0027] The third acquisition module is used to replace the preset information of the first position information in the first pose information with the depth information of each of the targets to obtain multiple second position information, and to perform coordinate system transformation on each of the second position information to obtain the corresponding pixel coordinate information.
[0028] The first determining module is used to match the pixel coordinate information and the region range information, and determine the target of interest among the multiple targets based on the matching result.
[0029] Thirdly, embodiments of this application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the target recognition method as described in any of the first aspects above.
[0030] Fourthly, embodiments of this application also provide a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the target recognition method as described in any of the first aspects above. Attached Figure Description
[0031] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0032] Figure 1 A schematic diagram of augmented reality glasses provided in one embodiment of this application;
[0033] Figure 2 A flowchart illustrating a target recognition method provided in one embodiment of this application;
[0034] Figure 3 A schematic diagram of a recognition scene based on AR glasses provided for one embodiment of this application;
[0035] Figure 4 A schematic diagram of the camera view in an AR glasses face recognition scenario provided in one embodiment of this application;
[0036] Figure 5 This is a schematic diagram of the structure of a target recognition system provided in one embodiment of this application;
[0037] Figure 6This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation
[0038] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0039] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, not limited in number; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.
[0040] The target recognition method provided in one embodiment of this application can be applied to smart wearable devices, such as AR (Augmented Reality) glasses, VR (Virtual Reality) glasses / headsets, MR (Mixed Reality) glasses / headsets, or other smart wearable devices. This application embodiment does not impose any limitations on the specific type of terminal device. The following description uses AR glasses as an example of a smart wearable device.
[0041] Figure 1 A schematic diagram of augmented reality glasses provided in one embodiment of this application can be referenced. Figure 1As shown, in current AR glasses applications, the camera module of the AR glasses is generally located at the upper left or upper right of the lens, which has a certain positional difference from the position of the human eye. This causes the virtual image (such as virtual information or virtual image) to be unable to align with the actual physical target seen by the human eye by default. For example, a user can see target A and target B through AR glasses, but for the user, the target of interest is target A, and the virtual information of target A needs to be displayed through AR glasses. However, due to the positional difference between the camera module and the position of the human eye, AR glasses may take target B as the user's target of interest, resulting in the incorrect display of the virtual information of target B. Therefore, it is necessary to perform virtual-real alignment or fusion processing on AR glasses. In addition, the field of view (FOV) of the display imaging module of AR glasses is relatively small (generally 30 to 40 degrees), which means that too much virtual imaging information cannot be displayed at the same time. It is necessary to accurately display the virtual content that the user cares about most on AR glasses.
[0042] One embodiment of this application provides an interactive method for user-initiated target recognition based on AR glasses, using a plane perpendicular to the wearer's face and located at the center of the line connecting the two eyes (see reference). Figure 1 The ray drawn from the center of the line connecting the left and right pupils allows AR glasses to accurately determine the target of interest actively identified by the user and display virtual information about that target. It should be noted that this application uses a face recognition scenario as an example. By transforming the coordinates of the glasses' camera and the center point of the glasses, accurate recognition of faces in the area of interest is achieved, effectively improving the user experience. The target recognition method provided in this application is also applicable to other target recognition scenarios (such as recognizing buildings, animals, etc.).
[0043] Figure 2 The flowchart illustrates a target recognition method according to one embodiment of this application. The steps of this method are merely one possible implementation of this application, including:
[0044] Step 201: Obtain the first pose information of the center point of the line connecting the eyes of the wearer of the smart wearable device.
[0045] In this embodiment of the application, after the user wears and turns on the AR glasses, the first pose information is obtained in real time, that is, the pose information (including position information and posture information) of the center point of the line connecting the user's eyes. Figure 3 This is a schematic diagram of a recognition scene based on AR glasses, provided as an embodiment of this application. (See attached diagram.) Figure 3As shown, the camera module on the lenses of the AR glasses has a large shooting range. Although the user's eyes can observe three target people in front, the user actively identifies only Pose3 (i.e., the target of interest). If the camera module simply judges the distance of each target person, it may lead to misidentification of the target of interest, causing the AR glasses to display virtual information of targets that the user is not interested in. Therefore, in performing virtual and real display of AR glasses, this embodiment first needs to obtain the pose information of the center point of the line connecting the user's eyes (Pose). ec (First pose information), that is, the center point of the line connecting the left and right eyes (such as the connection between the centers of the two eyeballs), and then the pose information is used to determine the pose. ec The user's actual interests are then determined through subsequent steps.
[0046] Step 202: Obtain the area range information and depth information of the multiple targets determined by the smart wearable device.
[0047] In this embodiment, the camera module of the AR glasses captures images of targets (people, animals, or buildings, etc.; this embodiment uses face recognition for illustration). The face recognition algorithm in the AR glasses is used to detect faces and mark region information, while simultaneously obtaining the location information corresponding to the face region in the face image. Figure 4 This is a schematic diagram of the camera view in a face recognition scene based on AR glasses, provided as an embodiment of this application. (See attached diagram.) Figure 4 As shown in this embodiment, the image captured by the camera module of the AR glasses involves three target face regions. These face regions are labeled using face bounding boxes, and the position information (pixel coordinates) of the face bounding boxes is obtained. For example, the range of the face region of face bounding box 401 is (u1, v1) to (u1', v1'), where u is the horizontal coordinate of the face bounding box, v is the vertical coordinate, |u1'-u1| is the length of face bounding box 401, and |v1'-v1| is the width of face bounding box 401. It should be noted that in this embodiment, existing models, such as the YOLO series and FaceNet, can be used to train a corresponding face recognition model with a dataset to recognize faces and obtain the face regions and their range information in the image captured by the camera module.
[0048] Furthermore, using artificial intelligence (AI) technology, depth detection is performed on the face scene within the current camera module's field of view to obtain face depth map information for each face region in the current face image. Then, the pixel coordinates of the previously obtained face regions are combined with the face depth map information to determine the depth information of each face region. For details, please refer to [reference needed]. Figure 4 In the context of Depth1, Depth2, and Depth3, it should be noted that in this embodiment, the depth information refers to the average depth value of the face region.
[0049] Step 203: Using the depth information of each target, replace the preset information of the first position information in the first pose information to obtain multiple second position information, and perform coordinate system transformation on each second position information to obtain the corresponding pixel coordinate information;
[0050] In this embodiment, it is necessary to align the virtual and real information of the region of actual interest to the user (i.e., the target face that the user is actually interested in) using the depth information and the first position information in the first pose information. Through the above steps, relevant information of multiple face regions that the user can observe has been obtained, including the pixel position coordinates and depth information of the face bounding box of each face region; then, it is necessary to further confirm which face region is the face region that the user needs to actively interact with (i.e., the face region that intersects with the normal radiating from the center point of the line connecting the user's eyes).
[0051] Furthermore, in this embodiment, the Z-axis coordinate value of the first position information in the first pose information is replaced by the depth information of each face recognition box, thereby obtaining multiple second position information, that is, obtaining the coordinate data to be converted corresponding to each face region, and then performing coordinate system transformation on each coordinate data to be converted to obtain the coordinate information corresponding to each coordinate data to be converted in the pixel coordinate system.
[0052] Step 204: Match the pixel coordinate information and the region range information, and determine the target of interest among the multiple targets based on the matching result.
[0053] In this embodiment, the pixel coordinate information of each face region is determined. If the pixel coordinate information corresponding to the face annotation box 401 is within the range of (u1, v1) to (u1', v1') (see reference). Figure 4If the target of interest is determined to be the target corresponding to face bounding box 401, then the pixel coordinate information corresponding to the next face bounding box (such as face bounding box 402) is matched with the area range information of the face bounding box until there is a pixel coordinate information corresponding to a face bounding box. If the pixel coordinate information is within the area range of the face bounding box, it is determined to be the target of interest.
[0054] The target recognition method provided in this application obtains the pixel coordinate information of each target by using the pose information of the center point of the line connecting the user's eyes and the depth information of each target. Then, the pixel coordinate information and the area range information are matched, and the target of interest actually observed by the user is determined from multiple targets based on the matching results. Finally, the virtual information corresponding to the target of interest is displayed through augmented reality glasses, thereby more accurately identifying the target area that the user is currently concerned with, completing the alignment operation between the real scene and the virtual content, and improving the user experience.
[0055] It should be noted that each implementation method of this application can be freely combined, rearranged, or executed individually, and does not need to rely on or depend on a fixed execution order.
[0056] In some embodiments, before obtaining the first pose information, the method further includes:
[0057] Determine whether a target image exists in front of the wearer's eyes;
[0058] If a single target image exists, display the information corresponding to that single target image.
[0059] In some embodiments, obtaining the first pose information includes:
[0060] If multiple target images exist, obtain the pose information of the center point of the line connecting the wearer's eyes.
[0061] In this embodiment, when a user uses AR glasses in a face recognition scenario, if only one target person is observed through the AR glasses, the information corresponding to that target person can be directly displayed after face recognition is completed. Only when multiple face images are determined to exist in front of the user will the pose information of the center point of the line connecting the user's eyes be collected, thereby achieving alignment between virtual and real content, improving the processing efficiency of the AR glasses, reducing their energy consumption, and ultimately enhancing the user experience.
[0062] In some embodiments, the method further includes:
[0063] Obtain information about the target of interest and display the information about the target of interest in the display area corresponding to the target of interest.
[0064] In this embodiment of the application, information about the target of interest, such as identity information corresponding to the target's facial image, can be displayed through the display module on the lens of the AR glasses based on relevant information about the target.
[0065] In some embodiments, obtaining the area range information of multiple targets determined by the smart wearable device includes:
[0066] The regions of each target are marked using a target detection algorithm to obtain the corresponding target regions and region range information.
[0067] In some embodiments, obtaining depth information of multiple targets determined by the smart wearable device includes:
[0068] Based on each of the target regions, obtain the corresponding target recognition box;
[0069] The depth information corresponding to each target region is determined based on the proportional mapping relationship between the height pixel value and the depth information of each target recognition box. The proportional mapping relationship is constructed by the height pixel value of the sample target recognition box and the sample depth information corresponding to the sample target recognition box.
[0070] In this embodiment, a face recognition function is built into the AR glasses. Face detection and region information labeling can be achieved through existing related technologies, such as open source solutions like the YOLO series and FaceNet. This function can be achieved by training with a dataset.
[0071] Furthermore, after identifying face bounding boxes for multiple targets, in some embodiments, to improve processing efficiency, the proportional relationship between depth information and the pixel values of the face bounding box height can be pre-constructed. That is, by pre-testing, pixel values of the same face bounding box height at different depths are collected in advance, and the scale relationship between pixels and depth is calculated. In the subsequent actual depth information calculation process, the average depth value of the face bounding box can be quickly obtained directly based on the pixel values of the face bounding box height.
[0072] In some embodiments, obtaining depth information of multiple targets determined by the smart wearable device includes:
[0073] Based on the depth sensor, the depth information corresponding to each of the target regions is obtained.
[0074] In this embodiment of the application, in order to further improve the accuracy of depth information, depth information of each face image region can be detected by artificial intelligence algorithms or depth sensors.
[0075] In some embodiments, the step of using the depth information of each of the targets to replace the preset information of the first position information in the first pose information to obtain multiple second position information, and performing coordinate system transformation on each of the second position information to obtain the corresponding pixel coordinate information, includes:
[0076] Based on the depth information, obtain the average depth value corresponding to each target area;
[0077] By replacing the coordinate values corresponding to the Z-axis of the first position information in the first pose information with each of the average depth values, multiple second position information are obtained;
[0078] After transforming multiple second position information from the world coordinate system to the camera coordinate system of the smart wearable device, and then transforming them from the camera coordinate system of the smart wearable device to the pixel coordinate system, the corresponding pixel coordinate information is obtained.
[0079] In the embodiments of this application, please refer to the appendix. Figure 3 The pose information of the center of the line connecting the user's eyes is Pose. ec (R ec |T ec ), that is, the first pose information, where R ec A 3x3 rotation matrix represents the angular change of the center of the eye-to-eye line relative to the initial moment, specifically:
[0080]
[0081] T ec =(X ec Y ec Z ec ), T ec A 3x1 matrix is used to represent the first position information, which characterizes the positional change of the center of the eye-to-eye line relative to the initial time, specifically:
[0082]
[0083] Furthermore, based on the depth information of multiple target regions (e.g., face bounding boxes 401, 402, and 403) obtained in the above embodiments, the coordinate value corresponding to the Z-axis of the position coordinate information (i.e., the first position information) in the first pose information is replaced, that is, the Z-axis value is changed. ec The replacement is performed to obtain the second position information.
[0084] For example, as shown in the appendix Figure 4 The depth information Depth1, Depth2, and Depth3 of the three targets shown are compared with the first position information T. ec Z-axis coordinates inec By performing the replacements respectively, we obtain three corresponding second position information:
[0085] T ec1 =(X ec Y ec ,Depth1);
[0086] T ec2 =(X ec Y ec ,Depth2);
[0087] T ec3 =(X ec Y ec Depth3).
[0088] It is understandable that the above three second position information T ec1 T ec2 and T ec3 The coordinate system is the world coordinate system. Furthermore, it is necessary to convert T... ec1 T ec2 and T ec3 These are then converted to pixel coordinates. More specifically, the three second position information T are first... ec1 T ec2 and T ec3 Transform to the camera coordinate system of the smart wearable device, and then transform the coordinates from the camera coordinate system to the pixel coordinate system. For example, using T... ec1 =(X ec Y ec Taking Depth1 as an example, the specific coordinate system transformation formula can be:
[0089]
[0090] Among them, the first 3x3 matrix This represents the intrinsic parameters of the camera in a smart wearable device. Clearly, for the three second location information T... ec1 T ec2 and T ec3 In the process of coordinate system transformation calculation, the intrinsic parameters of the smart wearable device camera and the matrix represented by the first pose information are the same for T. ec2 and T ec3 The coordinate system transformation calculation process can be found in T. ec1 The calculation process will not be elaborated here.
[0091] In some embodiments, matching the pixel coordinate information and the region range information, and determining the target of interest among a plurality of targets based on the matching result, includes:
[0092] The pixel coordinate information is matched with the corresponding region range information, and the target corresponding to the region range information including the pixel coordinate information is taken as the target of interest.
[0093] In this embodiment, a face detection algorithm is used to determine the location information of each face region, which can be referred to as... Figure 4 As shown, the region range information of face annotation box 401 is (u1, v1) to (u1', v1'), the region range information of face annotation box 402 is (u2, v2) to (u2', v2'), and the region range information of face annotation box 403 is (u3, v3) to (u3', v3'). Then, the pixel coordinate information obtained by coordinate system transformation in the above embodiment is matched one by one with the corresponding region range information, that is, the pixel coordinate information (u1, v1) is confirmed. ec1 v ec1 To confirm whether the pixel coordinates are within the range of (u1, v1) to (u1', v1'), please verify the pixel coordinate information. ec2 v ec2 Whether the face is within the range of (u2, v2) to (u2', v2'), and so on, until only one set of conditions is met, then the face area is the area where the user actively interacts, and the corresponding virtual information needs to be identified and displayed. In this embodiment, after determining the target of interest, visualization processing is performed through AR glasses, and then the virtual information is displayed at the corresponding position of the target of interest through the display module of the AR glasses.
[0094] The target recognition system provided in this application is described below. The target recognition system described below can be referred to in correspondence with the target recognition method described above.
[0095] Figure 5 This is a schematic diagram of the structure of a target recognition system provided in one embodiment of this application, as shown below. Figure 5 As shown in the figure, this application provides a target recognition system applied to a smart wearable device, including a first acquisition module 501, a second acquisition module 502, a third acquisition module 503, and a first determination module 504. The first acquisition module 501 is used to acquire the first pose information of the center point of the line connecting the eyes of the wearer of the smart wearable device; the second acquisition module 502 is used to acquire the region range information and depth information of multiple targets determined by the smart wearable device; the third acquisition module 503 is used to replace the preset information of the first position information in the first pose information with the depth information of each target to obtain multiple second position information, and to perform coordinate system transformation on each of the second position information to obtain the corresponding pixel coordinate information; the first determination module 504 is used to match the pixel coordinate information and the region range information, and determine the target of interest among the multiple targets based on the matching result.
[0096] The target recognition system provided in this application obtains the pixel coordinate information of each target by using the pose information of the center point of the line connecting the user's eyes and the depth information of each target. Then, it matches the pixel coordinate information with the area range information and determines the target of interest that the user actually observes from multiple targets based on the matching results. Finally, it displays the virtual information corresponding to the target of interest through augmented reality glasses, thereby more accurately identifying the target area that the user is currently concerned with, completing the alignment operation between the real scene and the virtual content, and improving the user experience.
[0097] In some embodiments, the system further includes a processing module and a virtual information display module, wherein the processing module is used to determine whether there is a target image in front of the wearer's eyes; and the virtual information display module is used to display virtual information corresponding to a single target image if a single target image exists.
[0098] The pose information acquisition module is specifically used to: if there are multiple target images, acquire the pose information of the center point of the line connecting the wearer's eyes.
[0099] In some embodiments, the second acquisition module includes a target detection unit, a bounding box annotation unit, and a depth information calculation unit. The target detection unit is used to mark the regions of each target using a target detection algorithm to obtain corresponding target regions and region range information. The bounding box annotation unit is used to obtain corresponding target bounding boxes based on each target region. The depth information calculation unit is used to determine the depth information corresponding to each target region based on the proportional mapping relationship between the height pixel values of each target bounding box and the depth information, wherein the proportional mapping relationship is constructed by the height pixel values of sample target bounding boxes and the sample depth information corresponding to the sample target bounding boxes.
[0100] In some embodiments, the second acquisition module is further configured to: obtain depth information corresponding to each of the target regions based on a depth sensor.
[0101] In some embodiments, the third acquisition module includes a depth information processing unit, a coordinate replacement unit, and a transformation unit. The depth information processing unit is used to acquire the average depth value corresponding to each target image region based on the depth information. The coordinate replacement unit is used to replace the coordinate value corresponding to the Z-axis of the first position information in the first pose information with each of the average depth values to obtain a plurality of second position information. The transformation unit is used to transform the plurality of second position information from the world coordinate system to the camera coordinate system of the smart wearable device, and then transform it from the camera coordinate system of the smart wearable device to the pixel coordinate system to obtain the corresponding pixel coordinate information.
[0102] In some embodiments, the first determining module includes a pixel coordinate matching unit, wherein the pixel coordinate matching unit is used to match the pixel coordinate information with the corresponding region range information, and to take the target corresponding to the region range information including the pixel coordinate information as the target of interest.
[0103] The system provided in this application is used to execute the above-described method embodiments. For specific processes and details, please refer to the above embodiments, which will not be repeated here.
[0104] Figure 6 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application, as shown below. Figure 6 As shown, the electronic device may include a processor 601, a communications interface 602, a memory 603, and a communication bus 604. The processor 601, communications interface 602, and memory 603 communicate with each other via the communication bus 604. The processor 601 can call logical instructions in the memory 603 to execute a target recognition method. This method includes: acquiring the first pose information of the center point of the line connecting the eyes of the wearer of the smart wearable device; acquiring the region range information and depth information of multiple targets determined by the smart wearable device; using the depth information of each target, replacing the preset information of the first position information in the first pose information to obtain multiple second position information, and performing coordinate system transformation on each of the second position information to obtain corresponding pixel coordinate information; matching the pixel coordinate information and the region range information, and determining the target of interest among the multiple targets based on the matching result.
[0105] Furthermore, the logical instructions in the aforementioned memory 603 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0106] On the other hand, one embodiment of this application also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer can execute the target recognition method provided by the above methods. The method includes: obtaining the first pose information of the center point of the line connecting the eyes of the wearer of the smart wearable device; obtaining the region range information and depth information of a plurality of targets determined by the smart wearable device; using the depth information of each of the targets, replacing the preset information of the first position information in the first pose information to obtain a plurality of second position information, and performing coordinate system transformation on each of the second position information to obtain the corresponding pixel coordinate information; matching the pixel coordinate information and the region range information, and determining the target of interest among the plurality of targets based on the matching result.
[0107] In another aspect, one embodiment of this application also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the target recognition method provided in the above embodiments. The method includes: acquiring first pose information of the center point of the line connecting the eyes of the wearer of the smart wearable device; acquiring region range information and depth information of a plurality of targets determined by the smart wearable device; using the depth information of each of the targets, replacing preset information of the first position information in the first pose information to obtain a plurality of second position information, and performing coordinate system transformation on each of the second position information to obtain corresponding pixel coordinate information; matching the pixel coordinate information and the region range information, and determining the target of interest among the plurality of targets based on the matching result.
[0108] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0109] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.
Claims
1. A target recognition method, characterized in that, Applications in smart wearable devices, including: Obtain the first pose information of the center point of the line connecting the eyes of the wearer of the smart wearable device; Obtain the area range and depth information of multiple targets determined by the smart wearable device; Using the depth information of each target, the preset information of the first position information in the first pose information is replaced to obtain multiple second position information, and the coordinate system is transformed for each second position information to obtain the corresponding pixel coordinate information; The pixel coordinate information and the region range information are matched, and the target of interest is determined among the multiple targets based on the matching result; The method involves using the depth information of each target to replace the preset information of the first position information in the first pose information to obtain multiple second position information, and performing coordinate system transformation on each of the second position information to obtain the corresponding pixel coordinate information, including: Based on the depth information, obtain the average depth value corresponding to each target area; By replacing the coordinate values corresponding to the Z-axis of the first position information in the first pose information with each of the average depth values, multiple second position information are obtained; After transforming multiple second position information from the world coordinate system to the camera coordinate system of the smart wearable device, and then transforming them from the camera coordinate system of the smart wearable device to the pixel coordinate system, the corresponding pixel coordinate information is obtained.
2. The target recognition method according to claim 1, characterized in that, The method further includes: Obtain information about the target of interest and display the information about the target of interest in the display area corresponding to the target of interest.
3. The target recognition method according to claim 1, characterized in that, The step of obtaining the area range information of multiple targets determined by the smart wearable device includes: The target detection algorithm is used to mark the regions of each target to obtain the corresponding target region and region range information.
4. The target recognition method according to claim 3, characterized in that, The step of obtaining depth information of multiple targets determined by the smart wearable device includes: Based on each of the target regions, obtain the corresponding target recognition box; The depth information corresponding to each target region is determined based on the proportional mapping relationship between the height pixel value and the depth information of each target recognition box. The proportional mapping relationship is constructed by the height pixel value of the sample target recognition box and the sample depth information corresponding to the sample target recognition box.
5. The target recognition method according to claim 3, characterized in that, The step of obtaining depth information of multiple targets determined by the smart wearable device includes: Based on the depth sensor, the depth information corresponding to each of the target regions is obtained.
6. The target recognition method according to claim 4 or 5, characterized in that, The step of matching the pixel coordinate information and the region range information, and determining the target of interest among multiple targets based on the matching result, includes: The pixel coordinate information is matched with the corresponding region range information, and the target corresponding to the region range information including the pixel coordinate information is taken as the target of interest.
7. A target recognition system, characterized in that, Applications in smart wearable devices, including: The first acquisition module is used to acquire the first pose information of the center point of the line connecting the eyes of the wearer of the smart wearable device. The second acquisition module is used to acquire the area range information and depth information of multiple targets determined by the smart wearable device; The third acquisition module is used to replace the preset information of the first position information in the first position information with the depth information of each of the targets to obtain multiple second position information, and to perform coordinate system transformation on each of the second position information to obtain the corresponding pixel coordinate information. The first determining module is used to match the pixel coordinate information and the region range information, and determine the target of interest among the multiple targets based on the matching result; The third acquisition module includes: A depth information processing unit is used to obtain the average depth value corresponding to each target image region based on the depth information. The coordinate replacement unit is used to replace the coordinate value corresponding to the Z-axis of the first position information in the first pose information with each of the average depth values to obtain multiple second position information. The transformation unit is used to transform multiple second position information from the world coordinate system to the camera coordinate system of the smart wearable device, and then transform them from the camera coordinate system of the smart wearable device to the pixel coordinate system to obtain the corresponding pixel coordinate information.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the target recognition method as described in any one of claims 1 to 6.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the target recognition method as described in any one of claims 1 to 6.