System and method for identifying identity through intelligent glasses and intelligent mobile terminal

Through a system that combines smart glasses with large language models, using face matching and voice reply functions, the problem of smart glasses not being able to identify identity in complex social scenarios is solved, and accurate confirmation of human identity and rich user experience are achieved.

CN118942457APending Publication Date: 2024-11-12SOLOS TECH SHENZHEN LTD
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411158265.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-21
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

Existing smart glasses cannot effectively identify and confirm the identity of people they know but do not contact much in complex social scenarios, and have a single function and a low degree of intelligence.

Method used

Take face images through smart glasses and combine large language models with cloud servers. Use contact photos in the album address book to match faces, and generate voice reply to confirm identity.

Benefits of technology

It realizes accurate identification of human identities in complex social scenarios, helps users get rid of the dilemma of not being able to remember human identities, enriches the functions of smart glasses, and improves the degree of intelligence.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118942457A_ABST
    Figure CN118942457A_ABST
Patent Text Reader

Abstract

The invention relates to a system and method for identifying identity through intelligent glasses and an intelligent mobile terminal.The system comprises the intelligent glasses, the intelligent mobile terminal connected with the intelligent glasses and a cloud server configured with a large language model, and the intelligent glasses control a camera to shoot a person in front of a user; a shot image and a first voice instruction of a user are sent to the intelligent mobile terminal, the intelligent mobile terminal carries out face matching on the image and a photo of a contact person in a photo album address book, and contact person data information of a target contact person corresponding to the matched photo and first prompt information are input into a large language model; and the large language model generates a first voice reply according to the contact information and the first prompt information, and sends the first voice reply to the intelligent glasses for playing through the intelligent mobile terminal. Through the system and method for identifying the identity through the intelligent glasses and the intelligent mobile terminal, the functionality of the intelligent glasses can be enriched, and the convenience of user operation is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of wearable devices, and in particular, to a system, method and smart mobile terminal for identifying identity through smart glasses. Background Art

[0002] With the development of computer technology, smart glasses are becoming more and more popular. Existing smart glasses usually have the functions of listening to music and making / receiving calls, with relatively simple functions and low intelligence.

[0003] Smart glasses have implemented some simple face recognition technologies, but in some complex application scenarios, for example, as people's social circles expand, they get to know more and more people, including business partners, classmates at training events, and social contacts based on relatives. But sometimes, when you meet someone you don't contact often, you may not remember their name. Currently, there is no technology for smart glasses to recognize the identity of acquaintances in the above scenarios, and the functions of smart glasses are not rich enough. Summary of the invention

[0004] The embodiments of the present application provide a system, method and smart mobile terminal for identifying identity through smart glasses. The smart glasses can be used to capture an image of a person and combine it with a large language model to identify the person in the address book.

[0005] On the one hand, an embodiment of the present application provides a system for identifying an identity through smart glasses, the system comprising:

[0006] Smart glasses, a smart mobile terminal connected to the smart glasses, and a cloud server configured with a large language model;

[0007] The smart glasses are used to control the camera of the smart glasses to take a picture of the person in front of the user, and send the taken image and the first voice command of the user to the smart mobile terminal, wherein the smart mobile terminal stores a photo album and a contact book, and the photo album and contact book include photos of the contacts and contact information corresponding to the photos;

[0008] The smart mobile terminal is used to perform face matching between the image and the photo of the contact in the album address book, and input the contact information of the target contact corresponding to the matched photo, the first text instruction converted from the first voice instruction, and the first prompt information generated based on the first voice instruction into the large language model;

[0009] The large language model is used to generate a first text reply to the first voice command according to the contact information of the target contact and the first prompt information, and return the first text reply to the smart mobile terminal;

[0010] The smart mobile terminal is used to send a first voice reply converted from the first text reply to the smart glasses;

[0011] The smart glasses are used to play the first voice response.

[0012] The present application also provides a method for identifying an identity through smart glasses, including:

[0013] The smart glasses control the camera of the smart glasses to take a photo of the person in front of the user, wherein the smart glasses are configured with a large language model and store a photo album and a contact list, wherein the photo album and the contact list include photos of the contacts and contact information corresponding to the photos;

[0014] The smart glasses perform face matching of the image with the photo of the contact in the album address book based on the first voice command of the user, and input the contact information of the target contact corresponding to the matched photo, the first text command converted from the first voice command, and the first prompt information generated based on the first voice command of the user into the large language model;

[0015] The large language model generates a first text reply to the first voice command based on the contact profile information of the target contact and the first prompt information; the smart glasses convert the first text reply into a first voice reply and play the first voice reply.

[0016] On one hand, an embodiment of the present application further provides a smart mobile terminal connected to smart glasses, wherein the smart mobile terminal is used to obtain an image sent by the smart glasses and a first voice command issued by a user wearing the smart glasses to the smart glasses, wherein the image is a photo of a person in front of the user taken by the smart glasses, and the smart mobile terminal is configured with a large language model and stores an album address book, wherein the album address book includes photos of contacts and contact information corresponding to the photos;

[0017] The smart mobile terminal is further used to perform face matching between the image and the photo of the contact in the album address book, and input the contact information of the target contact corresponding to the matched photo, the first text instruction converted from the first voice instruction, and the first prompt information generated based on the first voice instruction of the user into the large language model;

[0018] The large language model generates a first text reply to the first voice command according to the contact information of the target contact and the first prompt information;

[0019] The smart mobile terminal obtains the first text reply and converts it into a first voice reply, and sends the first voice reply to the smart glasses so that the smart glasses play the first voice reply.

[0020] It can be seen from the above-mentioned embodiments of the present application that the smart glasses use a camera to take a picture of the person in front of the user, and send the captured image and the user's first voice command to the smart mobile terminal. The smart mobile terminal matches the image with the photos of the contacts in the album address book stored locally in the smart mobile terminal, and inputs the contact information of the matched target contact and the prompt information generated based on the first voice command into the large language model of the cloud server. The large language model generates a voice reply for replying to the voice command based on the contact information and the prompt information, and sends the voice reply to the smart mobile terminal. The smart mobile terminal then sends the voice reply to the smart glasses for playback, so that the identity of a person can be judged through the smart glasses, helping users get rid of the dilemma of not being able to remember the identity of a person, enriching the functions of the smart glasses, and improving the intelligence of the smart glasses and the system for identifying identities through the smart glasses. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention.

[0022] Figure 1 A schematic diagram of the structure of a system for identifying identity through smart glasses provided in one embodiment of the present application;

[0023] Figure 2 A schematic diagram of the hardware structure of the smart glasses provided in one embodiment of the present application;

[0024] Figure 3 A schematic diagram of the hardware structure of smart glasses provided in another embodiment of the present application;

[0025] Figure 4 This is a schematic diagram of the architecture of a system for identifying identity through smart glasses in an embodiment of the present application;

[0026] Figure 5 A schematic diagram of the implementation flow of the method for identifying identity through smart glasses provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0028] See also Figure 1 , an embodiment of the present application provides a system for identifying identity through smart glasses, the system comprising:

[0029] Smart glasses 100, a smart mobile terminal 200 connected to the smart glasses 100, and a cloud server 300 configured with a large language model (LLM);

[0030] The smart glasses 100 are connected to the smart mobile terminal 200 in a wireless or wired manner, and the cloud server 300 is connected to the smart mobile terminal 200 in a wireless manner.

[0031] The smart glasses 100 are provided with a camera, a microphone, and a built-in nine-axis sensor. When the smart glasses 100 detect that a preset virtual button is pressed, or detects that the user uses a preset wake-up word, or detects a preset movement of the user's head, the shooting mode is turned on to control the camera to shoot the person in front of the user.

[0032] Among them, the preset virtual button can be set on the touch operation interface of the smart glasses 100;

[0033] The preset wake-up word may be, for example, "turn on shooting mode";

[0034] The preset action may be shaking or swinging the user's head according to preset rules, and the action may be recognized by the nine-axis sensor built into the smart glasses 100.

[0035] Turning on the shooting mode means automatically controlling the camera of the smart glasses 100 to shoot the person in front of the user. The smart glasses 100 sends the captured image together with the user's first voice command to the smart mobile terminal 200. The image includes at least the face image of the person and may further include a half-body image or a full-body image of the person.

[0036] The first voice command is obtained through the microphone of the smart glasses 100. The user can issue the first voice command before the camera of the smart glasses 100 takes a picture, or after the camera of the smart glasses 100 takes a picture, or at the same time as the camera of the smart glasses 100 takes a picture.

[0037] The first voice command includes information requesting confirmation of the identity of the person in front of the user. For example, the user issues the first voice command "Tell me the name of the person who just entered the hotel lobby. I seem to know him."

[0038] The smart mobile terminal 200 is provided with a voice-to-text module, which can convert the received voice information into text information, that is, convert the first voice instruction into a first text instruction.

[0039] The voice-to-text module can convert the voice command obtained by the smart mobile terminal 200, specifically the voice command obtained by the APP, into text and generate the prompt information. If the content of the voice command is not clear enough, a more accurate prompt information is generated by querying the historical records, so that the LLM can execute the voice command more accurately. The smart mobile terminal 200 stores an album address book, which includes photos of contacts and contact information corresponding to the photos, that is, each contact information in the album address book includes at least one facial photo of the contact and the contact information such as name, occupation, identity profile, contact number, social account, social account link and email corresponding to the facial photo.

[0040] The smart mobile terminal 200 has a built-in APP with a "contact image matching" function, which is used to match the captured image with the photos of the contacts in the album address book. When the APP is running, it can be set to only allow access to the album address book of the smart mobile terminal 200, while prohibiting access to other photos on the smart mobile terminal 200, and not storing or transmitting the images captured by the camera of the smart glasses 100, thereby improving the security of the system.

[0041] Specifically, the facial features in the image can be matched with the facial features in the photos of each contact, and the contact corresponding to the photo with the highest matching degree can be identified as the target contact, that is, the person in front of the user is identified as the target contact, and the contact information of the target contact is used as the contact information of the person. It supports identifying multiple people at a time.

[0042] The smart glasses 100 can control the camera to continuously take pictures of the person in front of the user, and continuously send the captured images to the smart mobile terminal 200. Correspondingly, the smart mobile terminal 200 continuously performs face detection on the acquired images to detect whether there is a face in the image; if a face is detected, the acquired image is continuously matched with the photos of the contacts in the album address book; if no face is detected, the further processing of the image is abandoned.

[0043] The smart mobile terminal 200 inputs the contact information of the target contact corresponding to the matched photo, the first text instruction converted from the first voice instruction, and the first prompt information generated based on the first voice instruction into the LLM;

[0044] LLM is a deep learning model trained with a large amount of text data that can generate natural language text or understand the meaning of language text. Large language models can handle a variety of natural language tasks such as text classification, question answering, dialogue, etc.

[0045] The first prompt information includes application example data learned by the LLM, and the application example data may be historical data. After learning the application example, the LLM may learn a data analysis and processing method similar to the content indicated by the first voice command, and may generate a first text reply based on the first voice command and the contact information of the target contact. For example, for the first voice command issued by the user, "Tell me the name of the person who just entered the hotel lobby, I seem to know him", a first text reply "His name is Chen Dawen, and he is the general manager of the Ecosystem Company" is generated;

[0046] The LLM returns the first text reply to the smart mobile terminal 200, and the smart mobile terminal 200 sends the first voice reply converted from the first text reply to the smart glasses 100, and the smart glasses 100 can play the first voice reply through the speaker.

[0047] The smart mobile terminal 200 is also provided with a text-to-speech module, which can convert the received voice reply in text format into a voice reply in audio format.

[0048] The text-to-speech module and the speech-to-text module may also be set on the cloud server 300 .

[0049] See also Figure 2 and Figure 3 The smart glasses 100 include: a frame 101, at least one temple 102, at least one microphone 103, at least one speaker 104, at least one processor 105, at least one memory 106, and at least one camera 107. The camera 107 can be Figure 3 As shown, it is arranged on the left side of the frame 101, and can also be arranged on the right side of the frame 101, or can also be arranged in the middle of the frame 101, and this application does not make any specific limitation. Figure 2 This is only a best example. In actual applications, the smart glasses 100 may also have Figure 2 Fewer or more components may be shown.

[0050] The frame 101 may be, for example, a front frame with a lens (eg, a sunglass lens, a transparent lens, or a corrective lens). The at least one temple 102 may include, for example, a left temple and a right temple.

[0051] The temple 102 is connected to the frame 101, and the processor 105 is electrically connected to the microphone 103, the speaker 104, the memory 106, and the camera 107. The microphone 103, the speaker 104, the processor 105, the memory 106, and the camera 107 are arranged on at least one temple 102 and / or the frame 101. Preferably, the temple 102 is detachably connected to the frame 101.

[0052] The smart glasses 100 also include a wireless communication component 108 electrically connected to the processor 105, and the wireless communication component 108 includes a wireless signal transceiver and its surrounding circuits, which can be specifically arranged in the inner cavity of the frame 101 and / or at least one temple 102. The wireless signal transceiver can be, but is not limited to, using at least one of the WiFi (Wireless Fidelity) protocol, NFC (Near Field Communication, near field communication) protocol, ZigBee (Purple Bee), UWB (Ultra Wideband, carrier-free communication), RFID (Radio Frequency Identification, Radio Frequency Identification) protocol and cellular mobile communication (Cellular Mobile Communication) protocol (such as 3G / 4G / 5G, etc.) for data transmission.

[0053] Optionally, in other embodiments of the present application, the smart glasses 100 further include a Bluetooth component 109 electrically connected to the processor 105;

[0054] The smart glasses 100 further include a battery 110 for providing power support for the electronic components of the smart glasses 100 , such as the microphone 103 , the speaker 104 , the processor 105 , the memory 106 , the camera 107 and other components.

[0055] The electronic components of the above-mentioned smart glasses can be connected through a bus.

[0056] It should be noted that the relationship between the components of the above-mentioned smart glasses can be a replacement relationship or a superposition relationship. That is, all the components in the above-mentioned embodiment can be installed on one smart glass, or a part of the above-mentioned components can be selectively installed according to needs. When it is a replacement relationship, the smart glasses are also provided with a connection interface for peripherals, and the connection interface can be at least one of a PS / 2 interface, a serial interface, a parallel interface, an IEEE1394 interface, a USB (Universal Serial Bus) interface, etc. The function of the replaced component can be realized by a peripheral connected to the connection interface, such as: an external speaker, an external sensor, etc.

[0057] Furthermore, the processor 105 includes: a central processing unit (CPU) and a DSP (Digital Signal Processing). The DSP is used to process the voice data acquired by the microphone 103. The CPU is preferably an MCU (Microcontroller Unit).

[0058] The memory 106 is a non-temporary memory, which may specifically include: RAM (Random Access Memory) and flash memory components, in which one or more programs executable by the processor 105 are stored, and the one or more programs include multiple instructions. The multiple instructions are used to:

[0059] The user's first voice command is obtained through the microphone 103, an image is captured through at least one camera 107, and sent to the smart mobile terminal 200 through the wireless communication component 108 or the Bluetooth component 109, the first voice response sent by the smart mobile terminal 200 is obtained, and the first voice response is played through the speaker 104.

[0060] Among them, the smart mobile terminal 200 receives the image taken by the camera and sent by the smart glasses 100, and performs face matching on the image with the photos of the contacts in the album address book locally saved by the smart mobile terminal 200, and inputs the contact information of the target contact corresponding to the matched photo, the first text instruction converted from the first voice instruction, and the first prompt information generated based on the first voice instruction into the LLM on the cloud server 300. The LLM generates a first text reply to the first voice instruction based on the contact information of the target contact and the first prompt information, and sends the first text reply to the smart mobile terminal 200.

[0061] The intelligent mobile terminal 200 may include, but is not limited to: a smart phone, a tablet computer, a cellular phone, a personal digital assistant (PDA) and other wireless communication devices. The intelligent mobile terminal 200 is also installed with Android, IOS or other operating systems.

[0062] The cloud server 300 may be a single server or a distributed server cluster consisting of multiple servers.

[0063] Among them, the cloud server 300 is configured with LLM, and LLM may include but is not limited to: Generative Artificial Intelligence Large Language Model (GAILLM) or Multimodal Large Language Model (MLLM).

[0064] The generative artificial intelligence large language model may be, for example, but not limited to: ChatGPT of Open AI, Bard of Google, and other models with similar functions. The multimodal large language model may be, for example, but not limited to: BLIP-2, LLaVA, MiniGPT-4, mPLUG-Owl, LLaMA-Adapter-v2, Otter, Multimodal-GPT, InstructBLIP, VisualGLM-6B, PandaGPT, LaVIN, and other models with similar functions. LLM may include multiple parts, which are trained based on different samples to answer different task requests issued by the smart mobile terminal 200, and may be, but not limited to: generating natural language voice replies, searching for emails, messages, photos, etc. by using tools, and calling network services and calling other third-party software development tools (Software Development Kit, SDK) to perform tasks provided by the SDK, etc.

[0065] The present embodiment provides a system for identifying identity through smart glasses. The smart glasses use a camera to take a picture of a person in front of a user, and send the taken image to a smart mobile terminal based on a first voice command of the user. The smart mobile terminal matches the image with a photo of a contact in an album address book stored locally in the smart mobile terminal, and inputs the contact information of the matched target contact, the text instruction converted from the voice instruction, and the prompt information generated based on the voice instruction into a large language model of a cloud server. The large language model generates a text reply for replying to the voice instruction based on the contact information and the prompt information, and sends the text reply to the smart mobile terminal. The smart mobile terminal converts the text into a voice reply and then sends the voice reply to the smart glasses for playback. Thus, the identity of a person can be determined through the smart glasses, helping the user get rid of the dilemma of not being able to remember the identity of the person, enriching the functions of the smart glasses, and improving the intelligence of the smart glasses and the system for identifying identity through the smart glasses.

[0066] In another embodiment, the user may continue to ask questions and issue a second voice command based on the first voice command.

[0067] The user issues a second voice command, which may be a further command based on the first voice response. For example, the user issues a second voice command "When was the last time we met? What was it about?"

[0068] The smart glasses 100 obtain the second voice command through the microphone, and send the second voice command to the smart mobile terminal 200. The smart mobile terminal 200 generates a second prompt message based on the second voice command. The smart mobile terminal 200 inputs the second text command converted from the second voice command, the second prompt message, and the most recent contact information into the LLM.

[0069] The second prompt information includes application example data corresponding to the second voice instruction for LLM to learn. After learning the application example, LLM can learn a data analysis and processing method similar to the content indicated by the second voice instruction, and can generate a second text reply based on the second voice instruction and the information of the target contact in the user's most recent contact information;

[0070] The LLM specifically searches for information related to the target contact in the user's most recent contact information according to the instruction of the second voice instruction;

[0071] The most recent contact information includes: emails, messages, photos, and other latest information generated when the user contacts other people.

[0072] The LLM generates a second text reply for the second voice command based on the searched information related to the target contact and the second prompt information, for example, for the second voice command issued by the user "When was the last time we met? What was it about?", a second text reply is generated "You had a business meeting in Japan three years ago to discuss potential partnerships";

[0073] The LLM returns the second text reply to the smart mobile terminal 200, and the smart mobile terminal 200 converts the second text reply into a second voice reply and sends it to the smart glasses 100, and the smart glasses 100 plays the second voice reply through the speaker.

[0074] The present embodiment provides a system for identifying identity through smart glasses. The smart glasses control a large language model through a smart mobile terminal according to the user's voice command to obtain the latest relevant information between the target contact in front of the user and the user from the user's most recent contact information, further enriching the functionality of the user to confirm the other party's identity through the smart glasses, providing users with higher convenience, and improving the intelligence level of the smart glasses and the system for identifying identity through the smart glasses.

[0075] In another embodiment, the user may continue to ask questions and issue a third voice command based on the first voice command or the second voice command.

[0076] The user issues a third voice command, which may be a further command based on the first voice response or the second voice response. For example, the user issues a third voice command "Can you tell me what he has been up to recently?"

[0077] The smart glasses 100 obtain the third voice command through the microphone, and send the third voice command to the smart mobile terminal 200. The smart mobile terminal 200 generates third prompt information based on the third voice command, and inputs the third text command converted from the third voice command and the third prompt information into the LLM.

[0078] The third prompt information includes application example data corresponding to the third voice command for LLM to learn. After LLM learns the application example, it can search for the latest network information of the target contact based on the third voice command and the contact profile information of the target contact, and generate a third voice reply;

[0079] The LLM specifically searches for the latest network information of the target contact according to the contact information of the target contact in accordance with the instruction of the third voice instruction;

[0080] The latest network information includes: the person's name, company, and other information used in social media (such as Facebook) or network searches at the most recent time.

[0081] LLM generates a third text reply for the third voice command based on the latest network information searched and the third prompt information. For example, for the third voice command issued by the user, "Can you tell me what he has been up to recently?", a third text reply is generated, "His company acquired another company, NovaNest, for $10 billion."

[0082] The LLM returns the third text reply to the smart mobile terminal 200, and the smart mobile terminal 200 sends the third voice reply converted from the third text reply to the smart glasses 100, and the smart glasses 100 plays the third voice reply through the speaker.

[0083] This embodiment provides a system for identifying identity through smart glasses. The smart glasses control a large language model on the Internet to search for the latest network information of a target contact based on the user's voice instructions through a smart mobile terminal, further enriching the functionality of the user confirming the other party's identity through the smart glasses, providing users with higher convenience, and improving the intelligence level of the smart glasses and the system for identifying identity through the smart glasses.

[0084] In another embodiment, based on the first voice command, if the smart mobile terminal 200 does not match the face in the image in the photo of the contact in the album address book, the social information of the user is obtained and the target contact is matched in the social information.

[0085] The social information includes a social image and social contact information corresponding to the social image. The social image includes photos of various social platforms. The social contact information includes contact information of the social contact, such as name, occupation, profile, contact number, social account, social account link and email address.

[0086] The smart mobile terminal 200 performs face matching between each social image of the user and the image captured by the camera. If a social image is matched, the social connection person corresponding to the matched social image is used as the target social contact. The target social contact is the person in front of the user. The person is not a contact in the user's photo album address book, but a contact of a contact of the user in the social image, such as friend B of the user's colleague A.

[0087] Inputting the social contact profile information of the target social contact corresponding to the matched social image, the first text instruction converted from the first voice instruction, and the first prompt information generated based on the first voice instruction into the LLM;

[0088] The LLM is used to generate a fourth text reply to the first voice command according to the social contact profile information of the target social contact and the first prompt information, and return the fourth text reply to the smart mobile terminal 200;

[0089] For example, the first voice command is "Tell me the name of the person who just entered the hotel lobby, I seem to know him", and the fourth text reply can be "His name is Zhang Jia, and he is a colleague of your friend Wang Yi".

[0090] The smart mobile terminal 200 sends the fourth voice reply converted from the fourth text reply to the smart glasses 100, and the smart glasses 100 play the fourth voice reply.

[0091] The present embodiment provides a system for identifying identity through smart glasses. Based on a voice command issued by the user to identify the identity of the person in front of the user, if the smart mobile terminal fails to match the face of the person in the image taken by the smart glasses in the album address book, the face of the person in the image is matched from the social image of the user, thereby realizing indirect identification of the identity of the user's social personnel, further enriching the functionality of the user to confirm the identity of a person through smart glasses, providing users with higher convenience, and improving the intelligence level of smart glasses and systems for identifying identities through smart glasses.

[0092] In another embodiment, based on the first voice command, if the smart mobile terminal 200 fails to match the face in the image with the photo of the contact in the album address book, the LLM is allowed to perform face matching through a third-party tool.

[0093] If the smart mobile terminal 200 does not match the face in the image with the photo of the contact in the album address book, it generates a fourth prompt message, and sends the image, the album address book, the first text instruction and the fourth prompt message to the LLM;

[0094] Among them, the fourth prompt information includes application example data corresponding to the first voice command for LLM to learn. After LLM learns the application example, the photos of the contacts in the album address book and the image can be sent to a third-party tool for face matching.

[0095] Specifically, according to the fourth prompt information, LLM sends the image and the album address book to a third-party tool for face matching, and obtains the matching result of the third-party tool.

[0096] The third-party tool may be other face matching models configured on the cloud server 300. Different matching algorithms have different accuracy and different requirements for image clarity. If no matching result is obtained on the smart mobile terminal 200, the third-party tool may be able to find a target contact in the album address book that matches the face in the image.

[0097] Further, if the third-party tool finds a target contact in the album address book whose contact photo matches the face of the image, the matching result is returned to the LLM;

[0098] The LLM generates the first text reply for the first voice command according to the contact information of the target contact and the first prompt information, and sends the first text reply to the smart mobile terminal 200;

[0099] The smart mobile terminal 200 sends the first voice reply converted from the first text reply to the smart glasses 100, and the smart glasses 100 play the first voice reply through the speaker.

[0100] The system for identifying identity through smart glasses provided in this embodiment is based on a voice command issued by the user to identify the identity of the person in front of the user. If the smart mobile terminal fails to match the face of the person in the image taken by the smart glasses in the album address book, the large language model is controlled to perform face matching through a third-party tool, thereby further enriching the functionality of users confirming the identity of people through smart glasses, providing users with higher convenience, and improving the intelligence level of smart glasses and systems for identifying identity through smart glasses.

[0101] Furthermore, if the smart mobile terminal 200 does not find a target contact in the album address book whose contact photo matches the face of the image, or if the third-party tool does not find a target contact in the album address book whose contact photo matches the face of the image, or if the smart mobile terminal does not find a target social contact in the social information whose social image matches the face of the image, the smart mobile terminal 200 issues a voice prompt to prompt the user to create a new contact profile based on the image taken by the camera.

[0102] For example, the content of the voice prompt is: He is not in the address book, do you want to add him to the address book?

[0103] The system for identifying identity through smart glasses provided in this embodiment is based on a voice command issued by the user to identify the identity of the person in front of the user. If the smart mobile terminal and the third-party tool are unable to match the face of the person in the image taken by the smart glasses in the album address book or social information, the user is reminded to add the person in front of the user to the album address book, further enriching the functionality of the user to confirm the identity of the person through smart glasses, providing users with higher convenience, and improving the intelligence of smart glasses and systems for identifying identity through smart glasses.

[0104] See also Figure 4 , Figure 4 The schematic diagram of the system architecture for identifying identity through smart glasses. The voice commands issued by the user are sent to the LLM through the smart terminal. The voice replies returned by the LLM based on each voice command are sent to the smart glasses through the smart terminal for playback.

[0105] Figure 4 The speech-to-text module and text-to-speech module are set on the cloud server. The speech-to-text module converts the voice command into text and sends it to the large language model. Each reply generated by the large language model is converted into an audio file that can be played on the smart glasses through the text-to-speech module.

[0106] It should be noted that the LLM can also be set on the smart mobile terminal 200, and the system for identifying identity through smart glasses includes smart glasses and a smart mobile terminal connected to the smart glasses. The data transmission between the smart mobile terminal and the LLM is changed from external transmission to internal transmission, and their functions are the same as those in the above embodiments.

[0107] The embodiment of the present application also provides a method for identifying an identity through smart glasses, and the identity of the person in front of the user is confirmed by the system for identifying an identity through smart glasses as described above. The application scenario of the method includes smart glasses. The APP and LLM of the "contact image matching" function in the above embodiment are both set on the smart glasses. The smart glasses in the embodiment of the present application are the smart glasses 100 in the above embodiment.

[0108] See also Figure 5 , Figure 5 The present invention is a flowchart of a method for identifying an identity through smart glasses, the method comprising:

[0109] S501, the smart glasses photograph the person in front of the user to obtain an image;

[0110] The smart glasses control the camera of the smart glasses to take a picture of the person in front of the user;

[0111] The smart glasses are configured with a large language model and store a photo album address book, which includes photos of contacts and contact information corresponding to the photos.

[0112] S502. Based on the first voice command, the smart glasses perform face matching on the image with the photo of the contact in the album address book, and input the contact information of the target contact corresponding to the matched photo, the first text command converted from the first voice command, and the first prompt information generated based on the first voice command into the large language model.

[0113] S503: The large language model generates a first text reply for the first voice command according to the contact information of the target contact and the first prompt information, and the smart glasses convert the first text reply into a first voice reply and play the first voice reply through the speaker.

[0114] For specific contents in the embodiments of this application, please refer to the description of the aforementioned embodiments.

[0115] In an embodiment of the present application, the smart glasses use a camera to take a picture of the person in front of the user, and the smart glasses perform face matching on the image with photos of contacts in the album address book stored locally on the smart glasses, and input the contact profile information of the matched target contact and the prompt information generated based on the voice command into the large language model of the cloud server. The large language model generates a voice response for replying to the voice command based on the contact profile information and the prompt information, and sends the voice response to the smart glasses for playback, so that the identity of a person can be determined through the smart glasses, helping users get rid of the dilemma of not being able to remember the identity of a person, enriching the functions of the smart glasses, and improving the intelligence of the smart glasses and the system for identifying identities through the smart glasses.

[0116] Further, the smart glasses obtain a second voice command of the user, and input the most recent contact information, a second text command converted from the second voice command, and second prompt information generated based on the second voice command into the large language model;

[0117] The large language model searches for information related to the target contact in the user's most recent contact information according to the instruction of the second voice instruction;

[0118] A second text reply for the second voice command is generated based on the searched information related to the target contact and the second prompt information; the smart glasses convert the second text reply into a second voice reply and play the second voice reply.

[0119] Furthermore, the smart glasses obtain a third voice instruction of the user, and input a third text instruction converted from the third voice instruction and third prompt information generated based on the third voice instruction into the large language model;

[0120] The large language model searches for the latest network information of the target contact according to the contact data information of the target contact in accordance with the instruction of the third voice instruction;

[0121] A third text reply to the third voice command is generated based on the latest network information searched and the third prompt information; the smart glasses convert the third text reply into a third voice reply and play the third voice reply.

[0122] Furthermore, if the smart glasses do not match the face in the image in the photo of the contact in the album address book, then obtain the social information of the user, the social information including the social image and the social contact profile information corresponding to the social image;

[0123] Performing face matching between the social image and the image captured by the camera;

[0124] and inputting the social contact profile information of the target social contact corresponding to the matched social image, the first text instruction converted from the first voice instruction, and the first prompt information generated based on the first voice instruction into the large language model;

[0125] The large language model generates a fourth text reply for the first voice command based on the social contact profile information of the target social contact and the first prompt information; the smart glasses convert the fourth text reply into a fourth voice reply and play the fourth voice reply.

[0126] Further, if the smart glasses do not match the face in the image in the photo of the contact in the album address book, a fourth prompt message is generated, and the image, the album address book and the fourth prompt message are sent to the large language model;

[0127] The large language model sends the image and the album address book to a third-party tool for face matching based on the fourth prompt information, and obtains the matching result of the third-party tool.

[0128] Furthermore, if the third-party tool finds a target contact in the album address book whose contact photo matches the face in the image, then

[0129] The large language model is used to generate the first text reply for the first voice command according to the contact information of the target contact and the first prompt information; the smart glasses convert the first text reply into a first voice reply and play the first voice reply.

[0130] Furthermore, the smart glasses are used to issue a voice prompt to prompt the user to create a new contact profile based on the image taken by the camera if the target contact whose contact photo matches the face of the image is not found in the album address book, or if the third-party tool does not find the target contact whose contact photo matches the face of the image in the album address book, or if the target social contact whose social image matches the face of the image is not found in the social information.

[0131] Furthermore, the smart glasses detect that a preset virtual button is pressed, or detect that the user uses a preset wake-up word, or detect a preset action of the user, and turn on the shooting mode.

[0132] Furthermore, the smart glasses control the camera of the smart glasses to continuously photograph the person in front of the user, and continuously perform face detection on the acquired image; if a face is detected, the acquired image is continuously matched with photos of contacts in the album address book; if no face is detected, further processing of the image is abandoned.

[0133] For other technical details not covered in this embodiment, please refer to the description in the aforementioned embodiment.

[0134] The embodiment of the present application also provides a smart mobile terminal, which is connected to the smart glasses. The APP and LLM of the "contact image matching" function in the above embodiment are set on the smart mobile terminal. The smart mobile terminal and smart glasses in the embodiment of the present application are respectively the smart mobile terminal 200 and smart glasses 100 in the above embodiment.

[0135] The smart mobile terminal is used to obtain an image sent by the smart glasses and a first voice command issued by the user to the smart glasses, wherein the image is a photo of a person in front of the user wearing the smart glasses taken by the smart glasses;

[0136] The smart mobile terminal is configured with a large language model and stores a photo album address book, the photo album address book including photos of contacts and contact information corresponding to the photos;

[0137] The smart mobile terminal is further used to perform face matching between the image and a photo of a contact in the album address book, and input the contact information of the target contact corresponding to the matched photo, the first text instruction converted from the first voice instruction, and the first prompt information generated based on the first voice instruction into the large language model, so that the large language model generates a first text reply to the first voice instruction according to the contact information of the target contact and the first prompt information;

[0138] The smart mobile terminal obtains the first text reply and converts it into a first voice reply, and sends the first voice reply to the smart glasses so that the smart glasses play the first voice reply.

[0139] Further, the intelligent mobile terminal is used to obtain a second voice command of the user, and input the most recent contact information, the second text command converted from the second voice command, the second prompt information generated based on the second voice command, and the most recent contact information into the large language model, so that the large language model is used to search for information related to the target contact in the most recent contact information of the user according to the instruction of the second voice command, and generate a second text reply to the second voice command according to the searched information related to the target contact and the second prompt information;

[0140] The smart mobile terminal obtains the second text reply and converts it into a second voice reply, and sends the second voice reply to the smart glasses so that the smart glasses play the second voice reply.

[0141] Further, the intelligent mobile terminal is used to obtain a third voice command of the user, and input a third text command converted from the third voice command and a third prompt information generated based on the third voice command into the large language model, and the large language model is used to search for the latest network information of the target contact according to the contact profile information of the target contact in accordance with the instruction of the third voice command, and generate a third text reply to the third voice command according to the searched latest network information and the third prompt information;

[0142] The smart mobile terminal obtains the third text reply and converts it into a third voice reply, and sends the third voice reply to the smart glasses so that the smart glasses play the third voice reply.

[0143] Furthermore, the smart mobile terminal is further used to obtain social information of the user if the face in the image is not matched in the photo of the contact in the album address book, the social information including the social image and the social contact profile information corresponding to the social image;

[0144] Performing face matching between the social image and the image captured by the camera;

[0145] and inputting the social contact profile information of the target social contact corresponding to the matched social image, the first text instruction converted from the first voice instruction, and the first prompt information generated based on the first voice instruction into the large language model, and the large language model is used to generate a fourth text reply to the first voice instruction according to the social contact profile information of the target social contact and the first prompt information;

[0146] The smart mobile terminal obtains the fourth text reply and converts it into a fourth voice reply, and sends the fourth voice reply to the smart glasses so that the smart glasses play the fourth voice reply.

[0147] Furthermore, the smart mobile terminal is also used to generate a fourth prompt message if the face in the image is not matched in the photos of the contacts in the album address book, and send the image, the album address book, the first text instruction, and the fourth prompt message to the large language model. The large language model sends the image and the album address book to a third-party tool for face matching based on the fourth prompt message, and the large language model obtains the matching result of the third-party tool.

[0148] Furthermore, if the third-party tool finds a target contact whose contact photo matches the face of the image in the album address book, the smart mobile terminal is further used to generate the first prompt information based on the first voice command, and input the first prompt information into the large language model, so that the large language model is used to generate the first text reply to the first voice command according to the contact profile information of the target contact and the first prompt information;

[0149] The smart mobile terminal obtains the first text reply and converts it into a first voice reply, and sends it to the smart glasses for playback.

[0150] Furthermore, if a target contact whose contact photo matches the face of the person in the image is not found in the album address book, or if the third-party tool does not find a target contact whose contact photo matches the face of the image in the album address book, or if a target social contact whose social image matches the face of the image is not found in the social information, the smart mobile terminal is further used to issue a voice prompt to prompt the user to create a new contact profile based on the image taken by the camera.

[0151] In an embodiment of the present application, the smart mobile terminal performs face matching on an image of a person in front of the user taken by the smart glasses and a photo of a contact in an album address book stored locally on the smart mobile terminal, and inputs the contact profile information of the matched target contact and the prompt information generated based on the voice command into a large language model of a cloud server, so that the large language model generates a voice response for replying to the voice command based on the contact profile information and the prompt information, obtains the voice response and sends the voice response to the smart glasses for playback, thereby enabling the smart mobile terminal to determine a person's identity, helping the user get rid of the dilemma of not being able to remember the person's identity, and improving the convenience of the user's operation of confirming the identity of the person in front of the user through the smart glasses.

[0152] For specific contents in the embodiments of this application, please refer to the description of the aforementioned embodiments.

[0153] It should be noted that, for the convenience of description, the aforementioned method embodiments are all described as a series of action combinations, but those skilled in the art should be aware that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0154] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0155] The above is a description of the system and method for identifying identity through smart glasses provided by the present invention. For those skilled in the art, according to the ideas of the embodiments of the present application, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A system for identifying identity through smart glasses, It is characterized in that The system comprises: Smart glasses, a smart mobile terminal connected to the smart glasses, and a cloud server configured with a large language model; The smart glasses are used to control the camera of the smart glasses to take a picture of the person in front of the user, and send the taken image and the first voice command of the user to the smart mobile terminal, wherein the smart mobile terminal stores a photo album and a contact book, and the photo album and contact book include photos of the contacts and contact information corresponding to the photos; The smart mobile terminal is used to perform face matching between the image and the photo of the contact in the album address book, and input the contact information of the target contact corresponding to the matched photo, the first text instruction converted from the first voice instruction, and the first prompt information generated based on the first voice instruction into the large language model; The large language model is used to generate a first text reply to the first voice command according to the contact information of the target contact and the first prompt information, and return the first text reply to the smart mobile terminal; The smart mobile terminal is used to send a first voice reply converted from the first text reply to the smart glasses; The smart glasses are used to play the first voice response.

2. The system according to claim 1, It is characterized in that The smart glasses are used to obtain a second voice command from the user and send the second voice command to the smart mobile terminal; The smart mobile terminal is used to generate second prompt information based on the second voice instruction, and input the most recent contact information, the second text instruction converted from the second voice instruction, and the second prompt information into the large language model; The large language model is used to search for information related to the target contact in the user's most recent contact information according to the instruction of the second voice instruction; generating a second text reply to the second voice command according to the searched information related to the target contact and the second prompt information, and returning the second text reply to the smart mobile terminal; The smart mobile terminal is used to send a second voice reply converted from the second text reply to the smart glasses; The smart glasses are used to play the second voice reply.

3. The system according to claim 1, It is characterized in that The smart glasses are used to obtain a third voice command from the user and send the third voice command to the smart mobile terminal; The intelligent mobile terminal is used to generate third prompt information based on the third voice instruction, and input the third text instruction converted from the third voice instruction and the third prompt information into the large language model; The large language model is used to search for the latest network information of the target contact according to the contact profile information of the target contact in accordance with the instruction of the third voice instruction; generating a third text reply for the third voice command according to the searched latest network information and the third prompt information, and returning the third text reply to the smart mobile terminal; The smart mobile terminal is used to send a third voice reply converted from the third text reply to the smart glasses; The smart glasses are used to play the third voice reply.

4. The system according to claim 1, It is characterized in that The smart mobile terminal is used to obtain the social information of the user if the face in the image is not matched in the photo of the contact in the album address book, wherein the social information includes a social image and social contact profile information corresponding to the social image; Performing face matching between the social image and the image captured by the camera; and inputting the social contact profile information of the target social contact corresponding to the matched social image, the first text instruction converted from the first voice instruction, and the first prompt information generated based on the first voice instruction into the large language model; The large language model is used to generate a fourth text reply to the first voice command according to the social contact profile information of the target social contact and the first prompt information, and return the fourth text reply to the smart mobile terminal; The smart mobile terminal is used to send a fourth voice reply converted from the fourth text reply to the smart glasses; The smart glasses are used to play the fourth voice reply.

5. The system according to claim 1, It is characterized in that The smart mobile terminal is used to generate fourth prompt information if the face in the image is not matched in the photo of the contact in the album address book, and send the image, the album address book, the first text instruction and the fourth prompt information to the large language model; The large language model is used to send the image and the album address book to a third-party tool for face matching according to the fourth prompt information, and obtain the matching result of the third-party tool.

6. The system according to claim 5, It is characterized in that The large language model is used to generate the first text reply to the first voice command according to the contact information of the target contact and the first prompt information if the third-party tool finds a target contact whose contact photo matches the face of the image in the album address book, and return the first text reply to the smart mobile terminal; The smart mobile terminal is used to send a first voice reply converted from the first text reply to the smart glasses; The smart glasses are used to play the first voice response.

7. The system according to claim 6, It is characterized in that The smart mobile terminal is used to issue a voice prompt to prompt the user to create a new contact profile based on the image taken by the camera if the target contact whose contact photo matches the face of the person in the image is not found in the photo album address book, or if the third-party tool does not find the target contact whose contact photo matches the face of the image in the photo album address book, or if the smart mobile terminal does not find the target social contact whose social image matches the face of the image in the social information.

8. The system according to claim 1, It is characterized in that The smart glasses are used to detect that a preset virtual button is pressed, or detect that the user uses a preset wake-up word, or detect a preset movement of the user's head, and turn on a shooting mode to control the camera to shoot a person in front of the user.

9. The system according to claim 1, It is characterized in that The smart glasses are used to control the camera of the smart glasses to continuously photograph the person in front of the user, and continuously send the photographed images to the smart mobile terminal; The intelligent mobile terminal is used to continuously perform face detection on the acquired image; If a face is detected, the acquired image is continuously matched with photos of contacts in the album address book; If no face is detected, further processing of the image is abandoned.

10. A method for identifying identity through smart glasses, It is characterized in that The method comprises: The smart glasses control the camera of the smart glasses to take a photo of the person in front of the user, wherein the smart glasses are configured with a large language model and store a photo album and a contact list, wherein the photo album and the contact list include photos of the contacts and contact information corresponding to the photos; The smart glasses perform face matching of the image with the photo of the contact in the album address book based on the first voice command of the user, and input the contact information of the target contact corresponding to the matched photo, the first text command converted from the first voice command, and the first prompt information generated based on the first voice command of the user into the large language model; The large language model generates a first text response to the first voice command based on the contact information of the target contact and the first prompt information; the smart glasses convert the first text response into a first voice response and play the first voice response.

11. The method according to claim 10, It is characterized in that The smart glasses acquire a second voice command of the user, and input the most recent contact information, a second text command converted from the second voice command, and second prompt information generated based on the second voice command into the large language model; The large language model searches for information related to the target contact in the user's most recent contact information according to the instruction of the second prompt information; and generates a second text reply to the second voice command based on the searched information related to the target contact and the second prompt information; The smart glasses convert the second text reply into a second voice reply and play the second voice reply.

12. The method according to claim 10, It is characterized in that The smart glasses acquire a third voice instruction of the user, and input a third text instruction converted from the third voice instruction and third prompt information generated based on the third voice instruction into the large language model; The large language model searches for the latest network information of the target contact according to the instruction of the third prompt information and the contact profile information of the target contact; generates a third text reply to the third voice command according to the searched latest network information and the third prompt information; the smart glasses convert the third text reply into a third voice reply and play the third voice reply.

13. The method according to claim 10, It is characterized in that If the smart glasses do not match the face in the image with the photo of the contact in the album address book, obtaining the social information of the user, the social information including the social image and the social contact profile information corresponding to the social image; Performing face matching between the social image and the image captured by the camera; and inputting the social contact profile information of the target social contact corresponding to the matched social image, the first text instruction converted from the first voice instruction, and the first prompt information generated based on the first voice instruction into the large language model; The large language model generates a fourth text reply to the first voice command based on the social contact profile information of the target social contact and the first prompt information; the smart glasses convert the fourth text reply into a fourth voice reply and play the fourth voice reply.

14. The method according to claim 10, It is characterized in that If the smart glasses do not match the face of the person in the image with the photo of the contact in the photo album address book, generate a fourth prompt message, and send the image, the photo album address book, the first text instruction and the fourth prompt message to the large language model; The large language model sends the image and the album address book to a third-party tool for face matching according to the fourth prompt information, and obtains the matching result of the third-party tool.

15. The method according to claim 14, It is characterized in that If the third-party tool finds a target contact in the album address book whose contact photo matches the face in the image, the large language model is used to generate the first text reply to the first voice command based on the contact profile information of the target contact and the first prompt information; the smart glasses convert the first text reply into a first voice reply and play the first voice reply.

16. The method according to claim 15, It is characterized in that The smart glasses are used to issue a voice prompt to prompt the user to create a new contact profile based on the image taken by the camera if the target contact whose contact photo matches the face of the person in the image is not found in the album address book, or if the third-party tool does not find the target contact whose contact photo matches the face of the image in the album address book, or if the target social contact whose social image matches the face of the image is not found in the social information.

17. A smart mobile terminal connected to smart glasses, It is characterized in that The smart mobile terminal is used to obtain an image sent by the smart glasses and a first voice command issued by a user wearing the smart glasses to the smart glasses, wherein the image is a photo of a person in front of the user taken by the smart glasses, and the smart mobile terminal is configured with a large language model and stores an album address book, wherein the album address book includes photos of contacts and contact information corresponding to the photos; The smart mobile terminal is further used to perform face matching between the image and the photo of the contact in the album address book, and input the contact information of the target contact corresponding to the matched photo, the first text instruction converted from the first voice instruction, and the first prompt information generated based on the first voice instruction of the user into the large language model; The large language model generates a first text reply to the first voice command according to the contact information of the target contact and the first prompt information; The smart mobile terminal obtains the first text reply and converts it into a first voice reply, and sends the first voice reply to the smart glasses so that the smart glasses play the first voice reply.

18. The intelligent mobile terminal according to claim 17, It is characterized in that The smart mobile terminal is used to obtain a second voice command of the user, and input the most recent contact information, a second text command converted from the second voice command, and second prompt information generated based on the second voice command into the large language model; The large language model is used to search the user's recent contact information for information related to the target contact according to the instruction of the second prompt information, and generate a second text reply to the second voice command according to the searched information related to the target contact and the second prompt information; The smart mobile terminal obtains the second text reply and converts it into a second voice reply, and sends the second voice reply to the smart glasses so that the smart glasses play the second voice reply.

19. The intelligent mobile terminal according to claim 17, It is characterized in that The smart mobile terminal is used to obtain a third voice instruction of the user, and input a third text instruction converted from the third voice instruction and third prompt information generated based on the third voice instruction into the large language model; The large language model is used to search the latest network information of the target contact according to the instruction of the third prompt information and the contact information of the target contact, and generate a third text reply to the third voice instruction according to the searched latest network information and the third prompt information; The smart mobile terminal obtains the third text reply and converts it into a third voice reply, and sends the third voice reply to the smart glasses so that the smart glasses play the third voice reply.

20. The intelligent mobile terminal according to claim 10, It is characterized in that The smart mobile terminal is further used to obtain social information of the user if the face in the image is not matched in the photo of the contact in the album address book, wherein the social information includes a social image and social contact profile information corresponding to the social image; Performing face matching between the social image and the image captured by the camera; and inputting the social contact profile information of the target social contact corresponding to the matched social image, the first text instruction converted from the first voice instruction, and the first prompt information generated based on the first voice instruction into the large language model; The large language model is used to generate a fourth text reply to the first voice instruction according to the social contact profile information of the target social contact and the first prompt information; The smart mobile terminal obtains the fourth text reply and converts it into a fourth voice reply, and sends the fourth voice reply to the smart glasses, so that the smart glasses play the fourth voice reply.

Citation Information

Cited By

  • System and method for identity recognition by means of smart glasses, and smart mobile terminal

    WO2026040652A1

  • Large language model-based information search system, smart glasses and information search method

    WO2026103435A1