Smart glasses system and smart glasses

Abstract

We provide a smart glasses system and smart glasses that enable the sharing of visual information. [Solution] The smart glasses system includes, for example, a first smart glasses and a second smart glasses. The first smart glasses include a camera that captures images in a direction including the user's field of view and generates image information, a first control unit that performs image recognition on the image information to extract objects and events, and a first communication unit that transmits visual information recording the content of the objects and events to one or more second smart glasses different from the first smart glasses. The second smart glasses include a second communication unit that receives visual information transmitted from the first smart glasses, and a second control unit that generates predetermined content about objects and events according to the content of the objects recorded in the visual information, and controls the second smart glasses to display the predetermined content.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to a smart glasses system and smart glasses.

BACKGROUND ART

[0002] Currently, smart glasses have come into widespread general use. Among information processing systems using such smart glasses, there is a system for supporting a user wearing the smart glasses to practice sports. The information processing system generates a moving video corresponding to a practice menu based on text, and displays the moving video on the smart glasses (see Patent Document 1).

PRIOR ART DOCUMENTS

PATENT DOCUMENTS

[0003]

PATENT DOCUMENT 1

SUMMARY OF THE INVENTION

MEANS FOR SOLVING THE PROBLEM

[0004] One embodiment of a smart glasses system comprises a plurality of smart glasses capable of communicating with each other, each of which comprises a camera, a communication unit, and a control unit, wherein the first smart glasses of the plurality comprises a camera that captures images of the direction including the user's field of view of the first smart glasses and generates image information, a first control unit which performs image recognition on the image information generated by the camera and extracts objects and events recorded in the image information, and a first communication unit which transmits visual information recording the contents of the objects and events extracted by the first control unit to one or more second smart glasses of the plurality that are different from the first smart glasses, wherein the second smart glasses comprises a second communication unit which receives visual information transmitted from the first communication unit of the first smart glasses, and a second control unit which generates predetermined content about objects and events according to the content of objects recorded in the visual information received by the second communication unit and controls the second smart glasses to display the predetermined content. [Brief explanation of the drawing]

[0005] [Figure 1] This is a diagram illustrating a smart glasses system according to one embodiment. [Figure 2] This is a block diagram illustrating a smart glasses according to one embodiment. [Figure 3] This is a block diagram illustrating the first smart glasses and the second smart glasses. [Figure 4] The first diagram illustrates an example of the criteria for determining the second type of smart glasses. [Figure 5] The second diagram illustrates an example of the criteria for determining the second smart glasses. (A) is a diagram illustrating the transmission of visual information from the first smart glasses to the second smart glasses and the feedback on the use of the visual information in the second smart glasses. (B) is a diagram illustrating the transmission of visual information from the first smart glasses to the second smart glasses after the process in (A). [Figure 6] This is the first diagram illustrating an example of redistribution (forwarding). [Figure 7] This is the second diagram illustrating an example of redistribution (forwarding). [Figure 8] This is a flowchart illustrating the outline of an information processing method according to one embodiment. [Modes for carrying out the invention]

[0006] One embodiment will be described below.

[0007] [Overview of Smart Glasses System 1] First, an overview of the smart glasses system 1 according to one embodiment will be described. Figure 1 is a diagram illustrating a smart glasses system 1 according to one embodiment.

[0008] Traditionally, smart glasses have been used to support sports training. By the way, smart glasses can be used for a variety of purposes, not just sports training. Therefore, the smart glasses system 1 of this embodiment provides a system that can share visual information such as objects and events acquired by one smart glasses 100 with other smart glasses 100.

[0009] The smart glasses system 1 of this embodiment comprises a plurality of smart glasses 100 (for example, a first smart glasses 100a and a second smart glasses 100b, etc. (see Figure 3)). The smart glasses 100 comprises, for example, a camera 121, a control unit 110, and a communication unit 122, etc. (see Figure 2). The control unit 110 may be equipped with, for example, artificial intelligence (AI). Each of the plurality of smart glasses 100 can communicate with each other, for example, by using the communication unit 122 provided in each unit. As an example, the smart glasses 100 (first smart glasses 100a) generates visual information by performing various processes on image information generated by capturing images with the camera 121 using the control unit 110, and transmits that visual information to another smart glasses 100 (second smart glasses 100b). Other smart glasses 100 (second smart glasses 100b) display warnings to the user based on visual information using augmented reality (AR), etc. (provides notifications (warning notifications) using AR, etc.). The smart glasses system 1 may also be a visual information sharing system using a smart glasses mesh comprising multiple smart glasses 100.

[0010] The smart glasses system 1 (a visual information sharing system using smart glasses mesh) may perform at least one of the following processes (1) to (8). (1) Images (still images or videos) are continuously captured by the camera 121 mounted on the smart glasses 100 (first smart glasses 100a). (2) The images captured in (1) above are analyzed to detect objects and events (visual information) that are depicted. (3) Select appropriate information from multiple visual information obtained according to the judgment algorithm implemented by the smart glasses 100 and broadcast it to other smart glasses 100 (second smart glasses 100b) in the vicinity. (4) Location information of the smart glasses 100 may be distributed along with visual information. (5) The other smart glasses 100 that have received the visual information decide whether to use the received information according to the criteria of the application being used by the device. (6) The other smart glasses 100 (second smart glasses 100b) generates and outputs at least one of the following from the received visual information: audio, image (video), and AR model (warning notification). (7) Other smart glasses 100 that have received the visual information will decide whether to redistribute it in accordance with the implemented redistribution algorithm. (8) Processing that requires resource capacity, such as image processing and AR rendering, may be offloaded to a mobile device (computer) such as a smartphone or tablet connected to the smart glasses 100 (or other smart glasses 100). The smart glasses system 1, by using a mesh system with multiple smart glasses 100, enables decision-making and processing with partial information, does not require other external equipment (other infrastructure) such as servers, is not limited by location, does not require large capital investment (equipment costs), and, for example, does not require the use of a WAN (Wide Area Network) line (wide area communication network), thus reducing the communication load.

[0011] For example, the smart glasses system 1 can extract visual information using the camera 121 and the AI-equipped smart glasses 100, and share that visual information with unspecified third parties in the surrounding area (other smart glasses 100). As a more specific example, one smart glass 100 (first smart glass 100a) generates image information by capturing an image with a camera 121 mounted on its own device, and when it is estimated that a "traffic accident" has occurred in response to analyzing and processing the image information by a control unit 110 (local AI) mounted on its own device, it generates visual information indicating that a traffic accident has occurred, and transmits the visual information to another smart glass 100 (second smart glass 100b) via a communication unit 122. When the other smart glass 100 receives the visual information via the communication unit 122, it outputs a caution notification by AR or the like. The caution notification is not limited to the AR-based caution notification, and may be a caution notification using images (still images and moving images), audio (sound), or the like.

[0012] Such a smart glass 100 may be, for example, a visual information sharing system using a smart glass mesh composed of a plurality of smart glasses 100. The smart glass system 1 shares, for example, not image information but visual information among the plurality of smart glasses 100 (transmits the visual information from one smart glass 100 to another smart glass 100), so that privacy of a person or the like recorded in the image information can be considered, and the communication volume can be reduced since the visual information having a smaller data amount than the image information is transmitted.

[0013] Note that "smart glass" can be rephrased as a "computer device" such as a "wearable terminal" and a "wearable device", for example. That is, the smart glass 100 may be, for example, a glasses-type computer device or the like. Accordingly, the "smart glass system" may be rephrased as a "computer device system" such as a "wearable terminal system" and a "wearable device system", for example. That is, the smart glass system 1 may be, for example, a computer system including a plurality of glasses-type computer devices or the like.

[0014] [Details of Smart Glass 100] Next, a smart device according to an embodiment will be described in detail. Figure 2 is a block diagram for explaining smart glasses 100 according to an embodiment.

[0015] Each of the plurality of smart glasses 100 includes, for example, a camera 121, a communication unit 122, a storage unit 123, a display unit 124, a control unit 110, and the like. The control unit 110 may be configured by, for example, an arithmetic processing unit of an information processing apparatus. The control unit 110 (e.g., an arithmetic processing unit or the like) may implement various processing functions by appropriately reading and executing various programs and the like stored in the storage unit 123 or the like, for example. That is, the processing functions may be implemented by computer implementation.

[0016] The camera 121 (camera unit) captures, for example, the field of view (at least a part of the field of view) of the user wearing the smart glasses 100, and generates image information.

[0017] The communication unit 122 is, for example, a communication interface capable of transmitting and receiving various information to and from an apparatus external to the information processing apparatus (external apparatus) and the like. The communication unit 122 is capable of communicating with one or a plurality of smart glasses 100, for example. Further, the communication unit 122 may be capable of receiving positioning satellite signals using a positioning satellite system such as GPS and GNSS, for example. This allows the smart glasses 100 to acquire position information and the like.

[0018] The storage unit 123 may store, for example, various information and programs. An example of the storage unit 123 may be a memory, a solid state drive, a hard disk drive, or the like.

[0019] The display unit 124 is, for example, a display capable of displaying various characters, symbols, images, and the like. The display unit 124 may be, for example, a lens surface or the like of the smart glasses 100.

[0020] The smart glasses 100 may be connected to a mobile device (not shown) such as a smartphone or tablet, and some of the processing functions of the control unit 110 may be implemented by the control unit (terminal control unit) (not shown) of the mobile device (offloading). In this case, the terminal control unit of the mobile device (for example, a processing unit, etc.) may implement various processing functions, such as an image recognition (analysis processing) function based on image information generated by the camera 121, an AR and image (still image and video) generation function for warning notifications, and a communication function with other smart glasses 100 (mobile devices connected to other smart glasses 100).

[0021] Multiple smart glasses 100 units, as described above, come together to form a smart glasses system 1. These multiple smart glasses 100 units can communicate with each other using their own communication units 122.

[0022] The smart glasses system 1 comprises multiple smart glasses 100.

[0023] (Details of the first smart glasses 100a) Next, we will describe the details of the first smart glasses 100a among the multiple smart glasses 100. Figure 3 is a block diagram illustrating the first smart glasses 100a and the second smart glasses 100b.

[0024] The first smart glasses 100a are smart glasses that serve as a source for transmitting visual information.

[0025] As shown in Figure 3 as an example, the first smart glasses 100a among the multiple smart glasses 100 includes a camera 121, a communication unit 122 (first communication unit 122a), a storage unit 123, a display unit 124, and a control unit 110 (first control unit 110a), etc.

[0026] Camera 121 captures images in the direction including the user's field of view of the first smart glasses 100a and generates image information. Camera 121 may, for example, capture video, or it may capture still images at predetermined intervals. Camera 121 may, for example, capture images in front of the user.

[0027] The first control unit 110a performs image recognition on image information generated by the camera 121, for example, and extracts objects and events recorded in the image information. The first control unit 110a may also perform image analysis based on the image information and obtain the contents of objects recorded in the image information. Here, an example of an object extracted by the first control unit 110a may be a person, a bicycle, a motorcycle, an automobile, an electric kick boat, etc. That is, the contents of an object may be, for example, the type of object (e.g., a person, a bicycle, a motorcycle, an automobile, etc.), whether the objects are the same or different, whether the objects are moving or not, if the objects are moving, the direction and speed of movement of the objects, the distance to the objects, etc. Also, an example of an event extracted by the first control unit 110a may be the occurrence of an accident, a dangerous driver, a pedestrian requiring consideration (elderly people, etc.), traffic congestion / density of people, etc. That is, the contents of an event may be, for example, the occurrence of an accident, a dangerous driver, a pedestrian requiring consideration (elderly people, etc.), traffic congestion / density of people, etc.

[0028] The first communication unit 122a transmits visual information, which records the contents of objects and events extracted by the first control unit 110a, to one or more second smart glasses 100b from among the multiple smart glasses 100 that are different from the first smart glasses 100a. In this case, the first communication unit 122a may broadcast the visual information to one or more second smart glasses 100b that are present around the first smart glasses 100a. That is, the first communication unit 122a may transmit the visual information to one or more second smart glasses 100b (unspecified second smart glasses 100b). Here, "broadcast" means, for example, transmitting to all second smart glasses 100b that are present within a predetermined range (the communication range of the first communication unit 122a).

[0029] (Priority determination) Next, we will explain how to determine (set) the priority when transmitting (distributing) visual information.

[0030] (Example 1 of priority determination) First, let's explain the first example of priority determination.

[0031] The first control unit 110a may, for example, extract one or more objects and events over time based on image information. If the objects extracted over time are the same object, the first control unit 110a may acquire the contents of the same object as movement characteristics, including the direction of movement of the same object and the distance from the first smart glasses 100a to that same object. The movement characteristics (content of the object (same object)) may, for example, as described above, be the type of object (person, bicycle, motorcycle, automobile, etc.), whether the objects are the same or different, whether the object is moving or not, if the object is moving, the direction and speed of movement of that object, the distance to the object, etc.

[0032] In this case, the first control unit 110a may, for example, determine the priority for transmitting visual information of the same object to the second smart glasses 100b according to its movement characteristics. For example, the first control unit 110a may group cases such as when the movement speed of the same object is relatively fast, when the distance from the first smart glasses 100a to the same object tends to decrease over time (when the same object approaches the first smart glasses 100a), and when the distance between the first smart glasses 100a and the same object is relatively short, and determine that the priority is relatively high if at least one of the cases in that group is met. Alternatively, the first control unit 110a may determine that the priority is higher if multiple cases in that group are met compared to when only one case is met.

[0033] The first control unit 110a may include the content and movement characteristics of the object to be extracted (acquired) in the visual information, as described above. The visual information may include, for example, the type of object (person, bicycle, motorcycle, automobile, etc.), whether the objects are the same or different, whether the objects are moving or not, the direction and speed of movement of the object if it is moving, the distance to the object, etc.

[0034] The first control unit 110a can, for example, use positioning satellite signals acquired by the communication unit 122 to acquire position information (first position information) indicating the position of the first smart glasses 100a (own device). The first control unit 110a may, for example, include first position information indicating the position of the first smart glasses 100a (own device) in the visual information and control the first communication unit 122a to transmit the visual information to the second smart glasses 100b according to priority. As an example, the first control unit 110a may control the first communication unit 122a to transmit visual information (for example, including first position information) of an object (the same object) that has been determined to have a relatively high priority to one or more second smart glasses 100b, which will be described later. As another example, the first control unit 110a does not have to transmit visual information of an object (the same object) that is determined to have a relatively low priority to the second smart glasses 100b, and may control the first communication unit 122a to transmit it to one or more second smart glasses 100b.

[0035] (Second example of priority determination) Next, we will explain the second example of priority determination.

[0036] As described above, the first control unit 110a may extract one or more objects continuously over time based on the image information. In this case, the first control unit 110a may acquire the contents of the objects as movement characteristics, including, for example, the direction of movement for each of the one or more objects extracted continuously over time, and the distance from the first smart glasses 100a to each of the one or more objects.

[0037] In this case, the first control unit 110a may, for example, when extracting multiple objects, use the movement characteristics of each of the multiple objects to determine which has a relatively higher priority based on at least one of the following: the distance from the first smart glasses 100a, the direction of movement of the object, and the speed of movement of the object. As a specific example, when the first control unit 110a extracts multiple objects, it may use the movement characteristics of each of the multiple objects to determine that at least one of the following has a relatively higher priority: an object that is closer to the first smart glasses 100a, an object that is approaching the first smart glasses 100a, and an object that is moving at a faster speed. That is, as an example, the first control unit 110a may group cases such as when an object is moving at a relatively fast speed, when the distance from the first smart glasses 100a to the object tends to decrease over time (when the same object approaches the first smart glasses 100a), and when the distance between the first smart glasses 100a and the object is relatively short, and determine that an object has a relatively higher priority if it falls into at least one of the cases in that group compared to when it does not fall into any of those cases.

[0038] The first control unit 110a may also control the first communication unit 122a to transmit visual information of an object with relatively high priority to the second smart glasses 100b. As an example, the first control unit 110a may control the first communication unit 122a to transmit visual information (e.g., including first position information) of an object determined to have a relatively high priority to one or more second smart glasses 100b, which will be described later. As another example, the first control unit 110a does not have to transmit visual information of an object that is determined to have a relatively low priority to the second smart glasses 100b, and may control the first communication unit 122a to transmit it to one or more second smart glasses 100b.

[0039] (Third example of priority determination) Next, we will explain the third example of priority determination.

[0040] The first control unit 110a may, for example, obtain second position information and movement direction information of the second smart glasses 100b from the second smart glasses 100b via the communication unit 122. The second position information may be, for example, information (position information) that records the position acquired by the second smart glasses 100b. The movement direction information may be, for example, information indicating the direction of movement of the second smart glasses 100b based on the movement trajectory of the position acquired by the second smart glasses 100b.

[0041] In this case, the first control unit 110a may determine the priority of an object based, for example, on the movement characteristics of the object acquired as described above, and the position and direction of movement of the second smart glasses 100b (the position of the second smart glasses 100b recorded in the second position information, and the direction of movement recorded in the direction of movement information). If the first control unit 110a determines the priority of an object as described above, it may control the first communication unit 122a to transmit visual information of the object with a relatively higher priority to the second smart glasses 100b.

[0042] As an example in this case, the first control unit may identify an object approaching the second smart glasses 100b based on the object's movement characteristics and the position and direction of movement of the second smart glasses 100b (the position of the second smart glasses 100b recorded in the second position information and the direction of movement recorded in the direction of movement information). The first control unit 110a may estimate the position of an object based on the first position information acquired by the first smart glasses 100a, and determine whether the object is approaching the second smart glasses 100b based on the estimated position of the object, the object's movement characteristics, and the position and direction of movement of the second smart glasses 100b. The first control unit 110a may, for example, determine that an object is approaching the second smart glasses 100b, and therefore the priority of that object approaching the second smart glasses 100b is relatively high. On the other hand, the first control unit 110a may, for example, determine that an object is moving away from the second smart glasses 100b, and therefore the priority of that object approaching the second smart glasses 100b is relatively low. In other words, the first control unit 110a may set the priority of an object approaching the second smart glasses 100b higher than the priority of an object moving away from the second smart glasses 100b. If the priority of objects is set as described above, the first control unit 110a may control the first communication unit 122a to transmit visual information of the object with a relatively high priority to the second smart glasses 100b.

[0043] (Details of the 2nd Smart Glasses 100b) Next, we will describe the details of the second smart glasses 100b, one of the multiple smart glasses 100. The second smart glasses 100b are smart glasses that serve as a recipient of visual information, or as a source or destination for transmitting visual information.

[0044] As illustrated in Figure 3, the second smart glasses 100b among the multiple smart glasses 100 includes a camera 121, a communication unit 122 (second communication unit 122b), a storage unit 123, a display unit 124, and a control unit 110 (second control unit 110b), etc.

[0045] The second communication unit 122b receives visual information transmitted from the first communication unit 122a of the first smart glasses 100a.

[0046] The second control unit 110b generates predetermined content about objects and events according to the content of objects recorded in the visual information received by the second communication unit 122b, and controls the second smart glasses 100b to display the predetermined content. That is, the second control unit 110b may, for example, determine whether an object is an object that requires attention based on the visual information (the content of objects recorded in the visual information) and the judgment criteria set in advance for the device. If the second control unit 110b determines that an object recorded in the visual information is an object that requires attention, it generates content indicating attention as predetermined content about the object and event. In this case, the second control unit 110b may, for example, generate predetermined content about objects and events (for example, content indicating attention) using a generation AI. The content indicating attention may be various types of content, such as AR, sound, and images (still images and videos), indicating that an object that requires attention exists, that an object that requires attention is approaching, and the direction and distance of movement of the object that requires attention. Furthermore, the specified content regarding an object is not limited to content indicating a warning about that object, as described above. It may also be content that conveys objects and events extracted by the first control unit 110a to the user. Specific examples include various types of content such as warnings, the presence or absence of crowds, and information about desired stores.

[0047] The second control unit 110b outputs warning content from the second smart glasses 100b. For example, if the content is a warning using AR and images, the second control unit 110b displays it on the lens surface (display unit 124) of the second smart glasses 100b. Also, if the content is a warning using voice (sound), the second control unit 110b outputs it from a speaker (not shown) located in the second smart glasses 100b.

[0048] The second control unit 110b can, for example, use positioning satellite signals acquired by the second communication unit 122b to acquire position information (second position information) indicating the position of the second smart glasses 100b (own device). The second control unit 110b may, for example, when it receives visual information transmitted by the first communication unit 122a, determine whether the same object is an object that requires attention based on the movement characteristics and first position information of the same object recorded in the visual information, and the second position information indicating the position of the user acquired by the second smart glasses 100b. The second control unit 110b may, for example, make various determinations, such as whether the object (same object) is approaching the second smart glasses 100b (user), whether the position of the object (same object) and the position of the second smart glasses 100b (user) are relatively close, whether the movement speed of the object (same object) is relatively fast, and if these conditions are met, it may determine that the object (same object) is an object that requires attention. The above-mentioned conditions for matching include, for example, when the object (the same object) is approaching the second smart glasses 100b (the user's device), when the position of the object (the same object) and the position of the second smart glasses 100b (the user's device) are relatively close, or when the movement speed of the object (the same object) is relatively fast. In this case, the second control unit 110b may, for example, when determining whether an object (the same object) is an object that requires attention, generate content indicating attention to the same object as predetermined content for the object and event, and control the system to display that content indicating attention, similar to the case described above.

[0049] Figure 4 is the first diagram illustrating an example of the criteria for determining whether the second smart glasses 100b are suitable.

[0050] As illustrated in Figure 4, when the first smart glasses 100a recognize two bicycles (bicycle A and bicycle B), it generates visual information A for bicycle A and visual information B for bicycle B, and distributes both visual information A and visual information B to multiple second smart glasses 100b in the surrounding area. Visual information A and visual information B may record the result of the first smart glasses 100a recognizing the object (bicycle), whether each of the two objects (bicycles) is approaching or moving away from the first smart glasses 100a, the distance between each of the two objects (bicycles) and the first smart glasses 100a, the position information of the first smart glasses 100a (first position information), and the direction of movement of the first smart glasses 100a.

[0051] As illustrated in Figure 4, each of the multiple second smart glasses 100b receives visual information A and visual information B. Based on various information about itself (for example, its location information (second location information), direction of movement, speed of movement, and map (map information), etc.) and the information recorded in visual information A and visual information B, the second smart glasses 100b determine whether to pay attention to bicycle A and bicycle B. One of the second smart glasses 100b, for example, determines that bicycle A (visual information A) is an object that requires attention because the user's vehicle and bicycle A are approaching each other. Similarly, the other of the second smart glasses 100b, for example, determines that bicycle B (visual information B) is an object that does not require attention because the user's vehicle and bicycle B are moving away from each other. The other smart glasses 100b, for example, determines that bicycle A (visual information A) and bicycle B (visual information B) are objects that do not require attention because the user's own device and bicycles A and B are moving further apart.

[0052] Figure 5 is a second diagram illustrating an example of the criteria for the second smart glasses 100b. Figure 5(A) is a diagram illustrating the transmission of visual information from the first smart glasses 100a to the second smart glasses 100b and the feedback on the use of the visual information in the second smart glasses 100b. Figure 5(B) is a diagram illustrating the transmission of visual information from the first smart glasses 100a to the second smart glasses 100b after the processing in Figure 5(A).

[0053] As illustrated in Figure 5(A), the first smart glasses 100a delivers visual information C and visual information D to the second smart glasses 100b, similar to the example shown in Figure 4. Visual information C is the visual information of bicycle C, and visual information D is the visual information of bicycle D. As illustrated in Figure 5(A), after receiving visual information C and visual information D, the second smart glasses 100b determines that bicycle D is approaching it based on visual information C and visual information D, and determines that attention should be paid to bicycle D. The second smart glasses 100b transmits the determination result that attention should be paid to bicycle D (that only visual information D of visual information C and visual information D should be used) to the first smart glasses 100a (feedback to the first smart glasses 100a). As illustrated in Figure 5(B), the first smart glasses 100a transmits visual information D from visual information C to the second smart glasses 100b based on feedback from the second smart glasses 100b (the second smart glasses 100b has determined that attention should be paid to bicycle D (meaning that only visual information D will be used out of visual information C and visual information D)).

[0054] (Redistribution of visual information) Next, we will explain an example of when the second smart glasses 100b transmits (redistributes) visual information to other second smart glasses 100b. Figure 6 is the first diagram illustrating an example of redistribution (forwarding).

[0055] As illustrated in Figure 6, the second control unit 110b may, for example, redistribute (transmit (transfer)) the visual information received from the first smart glasses 100a to other devices (other second smart glasses 100b) located around itself (the second smart glasses 100b). That is, the second control unit 110b may, for example, receive visual information transmitted (distributed) from the first smart glasses 100a and control the second communication unit 122b to transmit (redistribute) the visual information to one or more other second smart glasses 100b located around it. Similarly, the second control unit 110b of another device (another second smart glasses 100b) may, for example, receive visual information and repeatedly transmit (redistribute) that visual information to yet another device (yet yet another second smart glasses 100b) located around the other device (another second smart glasses 100b).

[0056] (First example of redistribution) Next, we will explain the first example of the redistribution mentioned above.

[0057] The second control units 110b of multiple second smart glasses 100b, that is, the second control unit 110b of the own unit (second smart glasses 100b) and other units (other second smart glasses 100b) (including even other units (even more second smart glasses 100b)), may record the transmission count in the visual information each time they transmit the same visual information. That is, the second control unit 110b may increment the transmission count recorded in the visual information each time it transmits the visual information. That is, when the second control unit 110b of each of the multiple second smart glasses 100b transmits (redistributes) visual information, it may increment the transmission count by 1 each time it transmits (redistributes) and record it in the visual information, and then transmit the visual information with the incremented transmission count recorded. The second control unit 110b may, for example, repeatedly transmit visual information to other second smart glasses 100b different from its own until the number of transmissions reaches a preset threshold (first threshold). That is, for example, if the number of transmissions that constitutes the first threshold is 5, the second control unit 110b may transmit visual information to other devices (other second smart glasses 100b) up to 5 transmissions, and may be prevented from transmitting visual information to other devices (other second smart glasses 100b) after 6 transmissions.

[0058] (Second example of redistribution) Next, I will explain the second example of redistribution.

[0059] The second control units 110b of multiple second smart glasses 100b, that is, the second control unit 110b of the own unit (second smart glasses 100b) and other units (other second smart glasses 100b) (including even other units (even more second smart glasses 100b)), may record the distance over which the visual information was transmitted each time they receive the visual information. In this case, the second control unit 110b of the own device (second smart glasses 100b) may record the location of the own device (second location information) in the visual information when transmitting visual information and transmit it to the other device (other second smart glasses 100b). When the second control unit 110b of the other device (other second smart glasses 100b) receives the visual information, it may obtain the distance between the location of the other device and the location of the own device based on the location of the other device (second location information) and the location of the own device (second location information) recorded in the visual information. The second control unit 110b of the other device (other second smart glasses 100b) may, for example, use the distance obtained as the transmission distance of the visual information. The second control unit 110b of the other device (other second smart glasses 100b) may, for example, record the transmission distance obtained as the visual information. In other words, the second control unit 110b of the own device (second smart glasses 100b) and the other device (other second smart glasses 100b) may, for example, record the transmission distance over which the visual information was transmitted when it is received. To put it another way, the second control unit 110b of each of the multiple second smart glasses 100b may, for example, record second position information in the visual information each time the same visual information is transmitted, and acquire the transmission distance each time the same visual information is received.

[0060] Furthermore, if the same visual information is transmitted (redistributed) multiple times, each second control unit 110b of the multiple second smart glasses 100b may acquire the total transmission distance (total transmission distance) calculated by summing up the multiple transmissions. If the transmission distance of the same visual information (total transmission distance, which is the sum of one or more transmission distances) exceeds a preset threshold (second threshold), each second control unit 110b of the multiple second smart glasses 100b may stop transmitting (redistributing) the same visual information to other second smart glasses 100b that are different from its own.

[0061] (Third example of redistribution) Next, I will explain the third example of redistribution.

[0062] The second control unit 110b of the own device (second smart glasses 100b) may acquire the position (second position information) and the direction of movement (direction of movement information) of another device (another second smart glasses 100b) from that device. In this case, the second control unit 110b of the own device may acquire the second position information and direction of movement information from the other device, or, when it receives visual information distributed by the other device (which includes the second position information and direction of movement information), it may acquire the second position information and direction of movement information from that visual information. That is, the second control unit 110b of the own device (second smart glasses 100b) may acquire the second position information about the position acquired by the other device (another second smart glasses 100b) and the direction of movement information indicating the direction of movement based on the movement trajectory of that position from the other device (another second smart glasses 100b). The second control unit 110b of the own device (second smart glasses 100b) may determine whether the object is something that should be noticed by another device (another second smart glasses 100b) based on the second position information and movement direction information of the other device, and the movement characteristics of the object recorded in the visual information. For example, the second control unit 110b of the own device (second smart glasses 100b) may perform various determinations, such as whether an object is approaching another device (another second smart glasses 100b), whether the distance between the object and the other device (another second smart glasses 100b) is relatively short, or whether the object is moving at a relatively fast speed, and if these conditions are met, it may determine that the object is an object that requires attention. The conditions for meeting the above conditions may include, for example, when the object is approaching another device (another second smart glasses 100b), when the position of the object and the position of the other device (another second smart glasses 100b) are relatively close, or when the object is moving at a relatively fast speed. In this case, the second control unit 110b of the own device (second smart glasses 100b) may redistribute (transmit) visual information to the other device (other second smart glasses 100b) if it determines that the object is something that the other device (other second smart glasses 100b) should pay attention to.

[0063] (Fourth example of redistribution) Next, we will explain the fourth example of redistribution. Figure 7 is a second diagram illustrating an example of redistribution (forwarding).

[0064] As illustrated in Figure 7, if there are multiple first smart glasses 100a (first smart glasses E, H) that have detected the same object, the second smart glasses 100b (second smart glasses F) can acquire visual information from each of the multiple first smart glasses 100a (first smart glasses E, H). Here, the second control unit 110b of the second smart glasses 100b (second smart glasses F) may determine whether the objects recorded in each of the multiple visual information (bicycles in the example shown in Figure 7) are the same or different, based on the location information recorded in each of the multiple visual information. If the objects recorded in each of the multiple visual information are the same, the second control unit 110b of the second smart glasses 100b (second smart glasses F) may transmit (redistribute (transfer)) one of the multiple visual information to another second smart glasses 100b (second smart glasses G).

[0065] (Grouping of 100 smart glasses) Next, we will explain the grouping of the Smart Glasses 100.

[0066] Each control unit 110 (first control unit 110a and second control unit 110b) of the multiple smart glasses 100 (first smart glasses 100a and second smart glasses 100b) may acquire the position information and the direction of movement information of each device. That is, each control unit 110 may acquire position information (first position information and second position information) indicating the position of the device (each device) acquired by each device (each smart glasses 100), and direction of movement information indicating the direction of movement of each device (direction of movement of each device) indicated by the trajectory of that position. The control unit 110 identifies smart glasses 100 that are within a predetermined distance range based on the location information of each device. That is, the control unit 110 determines whether multiple smart glasses 100 are present in the vicinity based on the location information of each device. If the control unit 110 determines that multiple smart glasses 100 are present in the vicinity, that is, if it identifies smart glasses 100 that are within a predetermined distance range, it sets the multiple smart glasses 100 into a single group and groups them together. Furthermore, the control unit 110 determines a representative from that group. In this case, the control unit 110 may, for example, determine the smart glasses 100 (first smart glasses 100a) that are closest to any object of interest (for example, the object with the highest priority as described above) from among the set group as the representative of the group. That is, the control unit 110 may set the smart glasses 100 that will be the representative of the group as the first smart glasses 100a. The first control unit 110a of the first smart glasses 100a may designate other smart glasses 100 that are not grouped as the second smart glasses 100b. That is, the first control unit 110a sets other smart glasses 100 that are not in the group as the second smart glasses 100b. The first control unit 110a may also transmit the visual information generated by the first smart glasses 100a to the second smart glasses 100b.

[0067] [Information Processing Methods] Next, an overview of the information processing method according to one embodiment will be described. Figure 8 is a flowchart illustrating an overview of an information processing method according to one embodiment.

[0068] In step ST101, the camera 121 of the first smart glasses 100a captures images in the direction including the user's field of view of the first smart glasses 100a and generates image information.

[0069] In step ST102, the first control unit 110a of the first smart glasses 100a performs image recognition on the image information generated in step ST101 and extracts objects and events recorded in the image information. The first control unit 110a may acquire the movement characteristics of the object and, according to those movement characteristics, set the priority for transmitting visual information in step ST103, which will be described later. The first control unit 110a may acquire location information (second location information) and direction of movement (direction of movement information) from the second smart glasses 100b, which will be described later, and set the priority for transmitting visual information in step ST103 according to the position of the second smart glasses 100b and the direction of movement of the second smart glasses 100b. The first control unit 110a may, after transmitting visual information to the second smart glasses 100b in step ST103 (described later), set a priority in response to feedback from the second smart glasses 100b.

[0070] In step ST103, the first communication unit 122a of the first smart glasses 100a transmits (distributes) visual information recording the contents of the object extracted in step ST102 to the second smart glasses 100b. In this case, the first communication unit 122a may broadcast the visual information to one or more second smart glasses 100b located around the first smart glasses 100a. The first communication unit 122a may transmit visual information of an object whose priority has been set relatively high in step ST102 to the second smart glasses 100b. The first communication unit 122a may transmit first location information indicating the position of the first smart glasses 100a, including this information in the visual information.

[0071] In step ST104, the second communication unit 122b of the second smart glasses 100b receives the visual information transmitted in step ST103.

[0072] In step ST105, the second control unit 110b of the second smart glasses 100b generates predetermined content about an object according to the content of the object recorded in the visual information received in step ST104, and controls the output of that predetermined content. The second control unit 110b may determine whether an object recorded in the visual information is an object that requires attention, and if it determines that it is an object that requires attention, it may control the unit to output a warning notification indicating that the object should be paid attention to. The second control unit 110b may transmit (redistribute) (transfer) visual information to other second smart glasses 100b that are different from its own unit. Multiple second smart glasses 100b may repeat redistribution until the total number of redistributions reaches a threshold (first threshold). Also, multiple second smart glasses 100b may repeat redistribution until the total transmission distance over which redistributions have been repeated reaches a threshold (second threshold). Furthermore, the second control unit 110b of the second smart glasses 100b may transmit (redistribute) visual information of an object that is of interest to other second smart glasses 100b.

[0073] Furthermore, if the first smart glasses 100a and the second smart glasses 100b (smart glasses 100) described above are in the vicinity of each other (for example, if they are within a predetermined distance range), multiple smart glasses 100 within that range may be grouped into one group. One smart glasses 100 in that group may be designated as the representative, and the smart glasses 100 designated as the representative may primarily transmit (distribute) visual information.

[0074] [About the program] The control unit 110 of the smart glasses 100 may be implemented as a function of a computer's processing unit or the like. The smart glasses 100 may implement processing functions using, for example, one control unit 110 (e.g., a processing unit), or it may implement processing functions in a distributed manner using multiple different control units 110 (e.g., processing units). The information processing program can implement processing functions in the smart glasses 100 (computer). The information processing program may be recorded on a computer-readable, non-temporary, tangible recording medium such as memory, a solid-state drive, a hard disk drive, or an optical disc. The storage medium may be rephrased as, for example, a non-temporary, tangible, computer-readable medium for storing the information processing program. The information processing program may also be transmitted online. The information processing program can be implemented into a product (computer program product) by the control unit 110 (for example, an arithmetic processing unit).

[0075] The smart glasses 100 can combine one or any combination of the above-mentioned processes. In this disclosure, the term "information" is used, but the term "information" can be replaced with "data," and the term "data" can be replaced with "information."

[0076] [Aspects of this embodiment, effects of each aspect, and examples of each aspect] Next, an embodiment of this model and the effects of each embodiment will be described. Note that the embodiments described below are examples as of the time of filing, and this embodiment is not limited to the embodiments described below. In other words, this embodiment is not limited to the embodiments described below, and may be realized by appropriately combining the parts described above. Furthermore, lower-level embodiments may be referenced in any of the higher-level embodiments. Furthermore, the effects of this embodiment described below are merely examples, and the effects achieved by each embodiment are not limited to those described below. Also, each embodiment may achieve, for example, at least one of the effects described below.

[0077] (Aspect 1) One embodiment of a smart glasses system comprises a plurality of smart glasses capable of communicating with each other, and each of the plurality of smart glasses is equipped with a camera, a communication unit and a control unit, The first smart glasses among the multiple smart glasses include a camera that captures images in the direction including the user's field of view of the first smart glasses and generates image information, a first control unit that performs image recognition on the image information generated by the camera and extracts objects and events recorded in the image information, and a first communication unit that transmits the visual information recording the contents of the objects and events extracted by the first control unit to one or more second smart glasses different from the first smart glasses among the multiple smart glasses. The second smart glasses include a second communication unit which is a communication unit that receives visual information transmitted from the first communication unit of the first smart glasses, and a second control unit which is a control unit that generates predetermined content about objects and events according to the content of objects recorded in the visual information received by the second communication unit, and controls the display of the predetermined content on the second smart glasses. This allows the smart glasses system to share visual information such as objects and events acquired by one smart glasses (first smart glasses) with other smart glasses (second smart glasses). Smart glasses systems can share information such as accident occurrences, dangerous drivers, pedestrians requiring special attention (e.g., the elderly), and traffic congestion / dense population density. Smart glasses systems can be used not only for pedestrian safety but also to assist with the safe operation of motorcycles and electric scooters. The smart glasses system can notify the user of the second smart glasses of predetermined content by displaying that content.

[0078] As an example of the use of a smart glasses system, if we consider the sharing of traffic information, the smart glasses system may perform the following processes (1) to (8). (1) The first smart glasses capture images using a camera mounted on the device. (2) The first smart glasses analyze the image described in (1) above and detect moving objects such as pedestrians, bicycles, motorcycles and automobiles. (3) The first smart glasses determine the movement characteristics of the detected moving object (e.g., whether it is moving, the direction of movement, and the distance) by performing the analysis described in (2) above in a continuous manner. (4) The first smart glasses determine the priority of the visual information to be delivered in (5) described later, according to the movement characteristics described in (3) above. (5) The first smart glasses broadcast visual information (e.g., information on moving objects in traffic, information on movement characteristics, information on the location of the device and information on the direction of movement of the device) to other smart glasses in the vicinity, according to the distribution priority set out in (4) above. (6) The second smart glasses (information receiver) determine whether to pay attention to the moving object detected by the first smart glasses based on visual information, the location information of the vehicle, the direction of movement of the vehicle, the speed of movement of the vehicle, and map information, etc. (7) When the second smart glasses determine that a moving object should be noted as described in (6) above, they generate and output audio, images (video or still image) or AR based on visual information, etc., indicating that the moving object should be noted. (8) The second smart glasses determine whether to redistribute the visual information to another second smart glasses according to its own redistribution algorithm, and if it determines to redistribute, it redistributes (rebroadcasts) the visual information. In (6) above, the second smart glasses may decide that they should pay attention to the moving object detected by the first smart glasses when the moving object approaches the moving object based on the direction of the moving vehicle's own movement and the direction of the moving object's movement. On the other hand, the second smart glasses may decide that they do not need to pay attention to the moving object detected by the first smart glasses when the moving vehicle moves away from the moving object based on the direction of the moving vehicle's movement and the direction of the moving object's movement. Similarly, in (6) above, the second smart glasses may decide that they do not need to pay attention to the moving object detected by the first smart glasses when the direction of the moving vehicle moves away from the direction of the moving object's movement.

[0079] Furthermore, in a smart glasses system, the smart glasses that deliver visual information can deliver it with location information attached. In this case, the smart glasses may refer to their own position (location information) and map information to identify the intersection that is relatively close to them (for example, the closest intersection). Intersections recorded in the map information may be assigned identification information to identify the intersection, etc. The identification information may be, for example, 16-bit identification information. Alternatively, the direction of movement of an object may be represented by 3 bits, etc., by representing the directions of north, northeast, east, southeast, south, southwest, west, and northwest (representing the approximate direction). A smart glasses system can represent the position and direction of movement using a total of 19 bits, for example, by representing the position of the smart glasses and the position of an object using intersection location information (16 bits) and the direction of movement of the object using approximate direction (3 bits). When representing coordinates using longitude and latitude, 32 bits are required for both longitude and latitude. This allows smart glasses systems to reduce the amount of visual and other information they need to deliver.

[0080] Furthermore, in a smart glasses system, the smart glasses may use generative AI to detect abnormal conditions based on visual information, generate text, generate images, and generate AR images. For example, the first smart glasses may use a generation AI to recognize image information and detect abnormal conditions such as "a vehicle traveling at high speed," "a vehicle that has overturned," and "a collision between vehicles" as the content of an object, and generate text that explains the abnormal condition. The first smart glasses may also include the generated text in the visual information and deliver it. As another example, the second smart glasses may use a generation AI to generate images and AR images from text recorded in the visual information, and use those images and AR images to visually notify the user of an anomaly (warning).

[0081] (Aspect 2) In one embodiment of the smart glasses system, the first communication unit of the first smart glasses may broadcast visual information to one or more second smart glasses located around the first smart glasses. This allows the smart glasses system to share visual information such as objects and events acquired by one smart glasses (first smart glasses) with other smart glasses (second smart glasses).

[0082] (Aspect 3) In one embodiment of the smart glasses system, the first control unit of the first smart glasses continuously extracts objects and events over time, and acquires the movement characteristics of the same object as its content, including the direction of movement of the same object among the extracted objects and the distance from the first smart glasses to that same object. Depending on its movement characteristics, the system may determine the priority for transmitting visual information of the same object to the second smart glasses, include first position information indicating the position of the user acquired by the first smart glasses in the visual information, and control the first communication unit to transmit the visual information to the second smart glasses according to the priority. This allows the smart glasses system to set priorities according to the movement characteristics of objects, for example. As a specific example, the first smart glasses may prioritize objects in the order of two-wheeled vehicles > bicycles > pedestrians, with the priority increasing as the condition becomes more frequent. As another specific example, the first smart glasses may prioritize objects in the order of approaching objects > moving away objects > stationary objects, with the priority increasing as the condition becomes more frequent. As yet another specific example, the first smart glasses may prioritize objects in the order of relatively close objects > relatively far objects, with the priority increasing as the condition becomes more frequent. The first smart glasses can set priorities according to, for example, one of the three specific examples of movement characteristics, or a combination of several of the three specific examples of movement characteristics. Furthermore, the first smart glasses can, for example, obtain the position and direction of movement of the second smart glasses, which are the recipients of the visual information (receiving feedback from the second smart glasses), and set priorities according to the position and direction of movement of the second smart glasses (feedback from the second smart glasses). Furthermore, if the first smart glasses receive feedback from the second smart glasses that are transmitting the visual information, for example, when the second smart glasses have outputted an alert based on the visual information (an alert has been output) (by broadcasting the result from the second smart glasses), the first smart glasses may set a higher priority for that visual information (the object corresponding to that visual information).

[0083] (Aspect 4) In one embodiment of the smart glasses system, when the second control unit of the second smart glasses receives visual information transmitted by the first communication unit, it may determine whether the same object is an object that requires attention based on the movement characteristics and first position information of the same object recorded in the visual information and the second position information indicating the position of the device acquired by the second smart glasses. If it determines that the same object is an object that requires attention, it may generate content indicating attention to the same object as predetermined content for the object and event, and control the system to display the content indicating attention. This allows the smart glasses system to share visual information such as objects and events acquired by one smart glasses (first smart glasses) with another smart glasses (second smart glasses), and the other smart glasses (second smart glasses) can then present (output) warnings to the user of the first smart glasses based on the visual information.

[0084] (Appendix 5) In one embodiment of the smart glasses system, when a plurality of objects are extracted, the first control unit of the first smart glasses may determine which of the plurality of objects has a relatively high priority based on the movement characteristics of each object, according to at least one of the distance from the first smart glasses, the direction of movement of the object, and the speed of movement of the object, and control the first communication unit to transmit the visual information of the object with the relatively high priority to the second smart glasses. This allows the smart glasses system to prioritize objects based on their movement characteristics, for example.

[0085] (Aspect 6) In one embodiment of the smart glasses system, the first control unit of the first smart glasses may acquire second position information about the position of the first smart glasses acquired by the second smart glasses, and direction of movement information indicating the direction of movement based on the movement trajectory of that position, and determine the priority of the object based on the movement characteristics of the object, the position of the second smart glasses recorded in the second position information, and the direction of movement recorded in the direction of movement information, and control the first communication unit to transmit visual information of the object with a relatively high priority to the second smart glasses. As a result, in a smart glasses system, the first smart glasses can, for example, obtain the position and direction of movement of the second smart glasses, which are the recipients of the visual information (receiving feedback from the second smart glasses), and set priorities according to the position and direction of movement of the second smart glasses (feedback from the second smart glasses).

[0086] (Aspect 7) In one embodiment of the smart glasses system, the second control unit of the second smart glasses may, upon receiving visual information transmitted from the first smart glasses, transmit the visual information to one or more other second smart glasses located around the second smart glasses. This allows the smart glasses system to retransmit (rebroadcast) the visual information received by the second pair of smart glasses to other second pairs of smart glasses.

[0087] (Pattern 8) In one embodiment of the smart glasses system, the second smart glasses and the second control unit of each other second smart glasses may record the transmission count of the visual information each time it is transmitted, and may repeatedly transmit the visual information to other second smart glasses different from their own until the transmission count reaches a preset threshold. This allows the smart glasses system to retransmit (rebroadcast) the visual information received by the second pair of smart glasses to other second pairs of smart glasses. In this case, the smart glasses system, for example, when retransmitting (rebroadcasting) received visual information, follows its own redistribution algorithm (for example, a flooding algorithm that stipulates that visual information broadcast by the system once will not be rebroadcast twice), but additional redistribution conditions can be added to improve efficiency. Examples of additional redistribution conditions that the smart glasses system can add include the number of redistributions, the total transmission distance, the use of location information for both the smart glasses and the detected objects, not retransmitting visual information that the second smart glasses did not use (did not output a warning for), and coordination with the priority set in the first smart glasses (if the priority of the visual information in the second smart glasses is relatively low, it will not be redistributed). The smart glasses system can use one of the above redistribution conditions (multiple conditions), or use a combination of multiple of the above redistribution conditions (multiple conditions).

[0088] (Aspect 9) In one embodiment of the smart glasses system, when a second smart glasses and the second control unit of each other second smart glasses receive visual information, they record the transmission distance over which the visual information was transmitted in the visual information, and when the total transmission distance, which is the sum of one or more such transmission distances, exceeds a preset threshold, they may stop transmitting the visual information to other second smart glasses different from their own. This allows the smart glasses system to retransmit (rebroadcast) the visual information received by the second pair of smart glasses to other second pairs of smart glasses.

[0089] (Aspect 10) In one embodiment of the smart glasses system, the second control unit of the second smart glasses may acquire second position information about the position acquired by the other second smart glasses and direction of movement information indicating the direction of movement based on the movement trajectory of that position from the other second smart glasses, and determine whether the object is something that the other second smart glasses should pay attention to based on the second position information, direction of movement information and the movement characteristics of the object recorded in the visual information, and if it is determined that the object is something that the other second smart glasses should pay attention to, it may transmit visual information to the other second smart glasses. As a result, in the smart glasses system, the second smart glasses can acquire the position and direction of movement of other second smart glasses from a different second smart glasses in advance, and recognize the movement characteristics of those second smart glasses. Subsequently, when the second smart glasses acquire visual information from the first smart glasses, they can compare the movement characteristics of the object recorded in that visual information with the movement characteristics of the other second smart glasses. When an object approaches the other second smart glasses, the second smart glasses rebroadcast the visual information to the other second smart glasses. This allows the other second smart glasses to output information to draw attention to the object based on the visual information. Similarly, other second smart glasses can redistribute visual information to yet another set of second smart glasses, just like the second smart glasses described above. In other words, multiple sets of second smart glasses can repeatedly redistribute visual information. On the other hand, other second smart glasses may, for example, choose not to redistribute their smart glasses to the other second smart glasses if the object moves away from the other second smart glasses, based on the movement characteristics of the other second smart glasses and the movement characteristics of the object recorded in the visual information. Furthermore, if multiple first smart glasses detect the same object, the second smart glasses can acquire visual information from each of the multiple first smart glasses. Here, the second smart glasses may determine whether the objects recorded in each of the multiple visual information are the same or different, based on the location information recorded in each of the multiple visual information. If the objects recorded in each of the visual information are the same, the second smart glasses can transmit (redistribute (transfer)) one of the multiple visual information to the other second smart glasses.

[0090] (Aspect 11) In one embodiment of the smart glasses system, each control unit of a plurality of smart glasses acquires position information indicating the position of the unit acquired by each unit, and direction information indicating the direction of movement of each unit, which is shown by the trajectory of that position. The control unit determines whether multiple smart glasses are present in the surrounding area, and if it determines that multiple smart glasses are present in the surrounding area, it groups the multiple smart glasses together, determines a representative for the group, sets the smart glasses that become the representative of the group as the first smart glasses, and the first control unit of the first smart glasses may designate the other smart glasses that have not been grouped as second smart glasses and transmit the visual information generated by the first smart glasses to the second smart glasses. In other words, when multiple smart glasses are present in the same area or at a relatively close distance (i.e., there are multiple smart glasses users), the smart glasses system can group together smart glasses that detect the same object, and the smart glasses representing that group can deliver visual information. In this case, the smart glasses system can recognize that multiple smart glasses are in the same area based on the object detected and the position and direction of movement of the smart glasses upon the initial delivery of visual information. In this case, the smart glasses system can, for example, designate the smart glasses closest to the detected object, or the smart glasses closest to the intersection, as the representative of the group. Furthermore, in a smart glasses system, other smart glasses that are not representative of a group can temporarily stop delivering visual information, or relatively reduce the frequency of visual information delivery. This allows the smart glasses system to prevent redundant distribution packets from flooding the mesh network, even when multiple smart glasses capable of delivering visual information based on the same object are present, thus suppressing an increase in communication traffic.

[0091] (Aspect 12) One embodiment of a smart glasses system comprises a plurality of smart glasses terminals, each of which includes smart glasses and a mobile terminal capable of communicating with the smart glasses, and each of the plurality of smart glasses terminals is equipped with a camera, a communication unit and a control unit. The first smart glasses terminal among the multiple smart glasses terminals includes a camera that captures images in the direction including the user's field of view and generates image information, a first control unit that performs image recognition on the image information generated by the camera and extracts objects and events recorded in the image information, and a first communication unit that transmits the visual information recording the contents of the objects and events extracted by the first control unit to one or more second smart glasses terminals different from the first smart glasses terminal among the multiple smart glasses terminals. The second smart glasses terminal includes a second communication unit which is a communication unit that receives visual information transmitted from the first communication unit of the first smart glasses terminal, and a second control unit which is a control unit that generates predetermined content about objects and events according to the content of the objects recorded in the visual information received by the second communication unit, and controls the display of the predetermined content on the second smart glasses terminal. As a result, even when some of the functions of the smart glasses described in the above-described embodiment are offloaded to a mobile device such as a smartphone or tablet, the smart glasses system can achieve the same effects as the smart glasses system described in the above-described embodiment.

[0092] (Aspect 13) One embodiment of smart glasses includes a camera that captures images in a direction including the user's field of view and generates image information; a first control unit that performs image recognition on the image information generated by the camera and extracts objects and events recorded in the image information; and a first communication unit that transmits visual information recording the contents of the objects and events extracted by the first control unit to one or more second smart glasses different from the first smart glasses which are the user. As a result, the smart glasses can achieve the same effects as the smart glasses system described in one embodiment above.

[0093] (Aspect 14) One embodiment of smart glasses includes a second communication unit that receives visual information transmitted from a first smart glasses different from itself, and a second control unit that generates predetermined content about objects and events using a generation AI according to the content of objects recorded in the visual information received by the second communication unit, and controls the second smart glasses, which is the own device, to display the predetermined content. As a result, the smart glasses can achieve the same effects as the smart glasses system described in one embodiment above.

[0094] By using the above-mentioned disclosure, we can contribute to achieving Sustainable Development Goal (SDG) 9, "Build resilient infrastructure, promote inclusive and sustainable industrialization and foster innovation." [Explanation of Symbols]

[0095] 1. Smart Glasses System 100 Smart Glasses 100a First Smart Glasses 100b Second Smart Glasses 110 Control Unit 110a First control unit 110b Second Control Unit 121 Camera 122 Communications Department 122a 1st Communication Department 122b 2nd Communication Department 123 Memory Department 123 indicates the section

Claims

1. A smart glasses system comprising multiple smart glasses capable of communicating with each other, each of which comprises a camera, a communication unit, and a control unit, Of the multiple smart glasses, the first smart glasses is The camera captures images of the direction including the user's field of view of the first smart glasses and generates image information, The first control unit, which is the control unit, performs image recognition on the image information generated by the camera and extracts objects and events recorded in the image information, The first communication unit, which acts as a communication unit, transmits the visual information, which records the contents of the object and event extracted by the first control unit, to one or more second smart glasses, which are different from the first smart glasses, among a plurality of smart glasses. Equipped with, The aforementioned second smart glasses, A second communication unit, which is a communication unit that receives visual information transmitted from the first communication unit of the first smart glasses, The second control unit, which acts as the control unit, generates predetermined content about the object and event according to the content of the object recorded in the visual information received by the second communication unit, and controls the display of said predetermined content on the second smart glasses, Equipped with Smart glasses system.

2. The first communication unit of the first smart glasses broadcasts visual information to one or more second smart glasses located around the first smart glasses. The smart glasses system according to claim 1.

3. The first control unit of the first smart glasses is The aforementioned objects and events are extracted over time, The movement characteristics of the identical object, including the direction and speed of movement of the identical object among the extracted objects, and the distance from the first smart glasses to the identical object, are obtained. Depending on the movement characteristics, the priority for transmitting visual information of the same object to the second smart glasses is determined. The first communication unit controls the first smart glasses to include the first position information indicating the position of the device acquired by the first smart glasses into the visual information and to transmit the visual information to the second smart glasses according to the priority. The smart glasses system according to claim 1.

4. The second control unit of the second smart glasses is Upon receiving the visual information transmitted by the first communication unit, the system determines whether the same object is an object that requires attention, based on the movement characteristics of the same object recorded in the visual information and the first position information, and the second position information indicating the position of the device acquired by the second smart glasses. When determining whether the aforementioned identical object is an object that requires attention, the system generates content indicating attention to the aforementioned identical object as predetermined content for the object and event, and controls the system to display the content indicating attention. The smart glasses system according to claim 3.

5. The first control unit of the first smart glasses is When multiple objects are extracted, the movement characteristics of each of the multiple objects are used to determine which has a relatively higher priority based on at least one of the following: the distance from the first smart glasses, the direction of movement of the object, and the speed of movement of the object. The first communication unit is controlled to transmit visual information of the object with relatively high priority to the second smart glasses. The smart glasses system according to claim 3.

6. The first control unit of the first smart glasses is The second smart glasses acquire second position information regarding the position of the self-propelled device and movement direction information indicating the direction of movement based on the movement trajectory of said position. Based on the movement characteristics of the object, the position of the second smart glasses recorded in the second position information, and the direction of movement recorded in the movement direction information, the priority of the object is determined. The first communication unit is controlled to transmit visual information of the object with relatively high priority to the second smart glasses. The smart glasses system according to claim 4.

7. The second control unit of the second smart glasses is Upon receiving visual information transmitted from the first smart glasses, the second smart glasses transmit the visual information to one or more other second smart glasses located around them. The smart glasses system according to claim 4.

8. The second smart glasses and the second control unit of each of the other second smart glasses are: Each time visual information is transmitted, the transmission count of that visual information is recorded. The device can repeatedly transmit visual information to a second smart glasses device other than its own until the number of transmissions reaches a predetermined threshold. The smart glasses system according to claim 7.

9. The second smart glasses and the second control unit of each of the other second smart glasses are: When visual information is received, the transmission distance over which the visual information was transmitted is recorded in the visual information. When the total transmission distance, which is the sum of one or more such transmission distances, exceeds a preset threshold, the transmission of visual information to a second smart glasses device different from the current device will stop. The smart glasses system according to claim 7.

10. The second control unit of the second smart glasses is Second location information regarding the position acquired by the other second smart glasses, and direction of movement information indicating the direction of movement based on the movement trajectory of the said position are acquired from the other second smart glasses. Based on the second position information and direction of movement information, and the movement characteristics of the object recorded in the visual information, it is determined whether the object is something that the other second smart glasses should pay attention to. If the other second smart glasses determine that the object is something that requires attention, visual information is transmitted to the other second smart glasses. The smart glasses system according to claim 7.

11. Each of the control units of the multiple smart glasses is: Each aircraft acquires position information indicating its own position and direction information indicating the direction of movement of each aircraft, as shown by the trajectory of that position. Determine if multiple smart glasses are present in the surrounding area. If it is determined that multiple smart glasses are present in the vicinity, the multiple smart glasses are grouped together, and a representative of the group is determined. The smart glasses representing the group are designated as the first smart glasses. The first control unit of the first smart glasses designates other smart glasses that have not been grouped as the second smart glasses. The visual information generated by the first smart glasses is transmitted to the second smart glasses. The smart glasses system according to claim 1.

12. A smart glasses system comprising a plurality of smart glasses terminals, each of which includes smart glasses and a mobile terminal capable of communicating with the smart glasses, wherein each of the plurality of smart glasses terminals is equipped with a camera, a communication unit and a control unit, Of the multiple smart glasses terminals, the first smart glasses terminal is The camera captures images of the direction including the user's field of view of the first smart glasses terminal and generates image information, The first control unit, which is the control unit, performs image recognition on the image information generated by the camera and extracts objects and events recorded in the image information, The first communication unit is a communication unit that transmits the visual information, which records the contents of the objects and events extracted by the first control unit, to one or more second smart glasses terminals, which are different from the first smart glasses terminal, among a plurality of smart glasses terminals. Equipped with, The aforementioned second smart glasses terminal is A second communication unit which is a communication unit that receives visual information transmitted from the first communication unit of the first smart glasses terminal, The second control unit, which acts as the control unit, generates predetermined content about the object and event according to the content of the object recorded in the visual information received by the second communication unit, and controls the display of said predetermined content on the second smart glasses terminal, Equipped with Smart glasses system.

13. A camera that captures images in the direction including the user's field of view and generates image information, A first control unit performs image recognition on image information generated by the aforementioned camera and extracts objects and events recorded in the image information, A first communication unit transmits visual information, which records the contents of the objects and events extracted by the first control unit, to one or more second smart glasses different from the first smart glasses which are the own devices. Smart glasses equipped with [a specific feature].

14. A second communication unit that receives visual information transmitted from the first smart glasses, A second control unit generates content indicating predetermined information about the object and event using AI, according to the content of the object recorded in the visual information received by the second communication unit, and controls the second smart glasses, which are the user's device, to display the predetermined content. The smart glasses according to claim 12, comprising:

Images