Smart Glass System and Smart Glass

Abstract

Provided are a smart glass system and a smart glass capable of sharing visual information. 【Solution means】 The smart glass system includes, for example, a first smart glass and a second smart glass. The first smart glass includes a camera that images a direction including the user's field of view to generate image information, a first control unit that performs image recognition on the image information to extract objects and events, and a first communication unit that transmits visual information recording the content of the objects and events to one or more second smart glasses different from the first smart glass. The second smart glass includes a second communication unit that receives the visual information transmitted from the first smart glass, and a second control unit that generates predetermined content about the objects and events according to the content of the objects recorded in the visual information and controls to display the predetermined content on the second smart glass.

Description

Technical Field

[0001] The present disclosure relates to a smart glass system and smart glasses.

Background Art

[0002] Currently, smart glasses have become generally widespread. Among information processing systems that utilize such smart glasses, there is a system for assisting a user wearing the smart glasses in practicing sports. The information processing system generates a video corresponding to a practice menu based on text and displays the video on the smart glasses (see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Means for Solving the Problems

[0004] A smart glass system according to one aspect includes a plurality of smart glasses capable of communicating with each other. Each of the plurality of smart glasses includes a camera, a communication unit, and a control unit. The first smart glass among the plurality of smart glasses includes a camera that captures an image in a direction including the field of view of the user of the first smart glass to generate image information, a first control unit as a control unit that performs image recognition on the image information generated by the camera and extracts objects and events recorded in the image information, and a first communication unit as a communication unit that transmits visual information recording the contents of the objects and events extracted by the first control unit to one or more second smart glasses different from the first smart glass among the plurality of smart glasses. The second smart glass includes a second communication unit as a communication unit that receives the visual information transmitted from the first communication unit of the first smart glass, and a second control unit as a control unit that generates predetermined contents about the objects and events according to the contents of the objects recorded in the visual information received by the second communication unit and controls to display the predetermined contents on the second smart glass.

Brief Description of the Drawings

[0005]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

[0006] Hereinafter, an embodiment will be described.

[0007] [Outline of Smart Glass System 1] First, an outline of a smart glass system 1 according to an embodiment will be described. FIG. 1 is a diagram for explaining a smart glass system 1 according to an embodiment.

[0008] Conventionally, smart glasses have been used to assist in sports practice. By the way, smart glasses are considered to be applicable not only to sports practice but also to various applications. Therefore, the smart glass system 1 of this embodiment provides a system capable of sharing visual information such as objects and events acquired by one smart glass 100 with other smart glasses 100.

[0009] The smart glass system 1 of this embodiment includes a plurality of smart glasses 100 (for example, the first smart glass 100a, the second smart glass 100b, etc. (see FIG. 3)). The smart glass 100 includes, for example, a camera 121, a control unit 110, a communication unit 122, etc. (see FIG. 2). The control unit 110 may be equipped with, for example, artificial intelligence (AI: Artificial Intelligence). Each of the plurality of smart glasses 100 can communicate with each other using, for example, the communication unit 122 arranged in each device. As an example, the smart glass 100 (the first smart glass 100a) generates visual information by performing various processes on the image information generated by imaging with the camera 121 by the control unit 110, and transmits the visual information to another smart glass 100 (the second smart glass 100b). Another smart glass 100 (the second smart glass 100b) displays an alert on the own device based on the visual information (performs a notification (alert notification) using AR or the like). The smart glass system 1 may be a visual information sharing system using a smart glass mesh including a plurality of smart glasses 100.

[0010] The smart glass system 1 (the visual information sharing system using a smart glass mesh) may perform at least one of the following multiple processes (1) to (8). (1) Continuously capture images (still images or moving images) with the camera 121 mounted on the smart glass 100 (the first smart glass 100a). (2) Analyze the image captured in (1) above and detect the objects and events (visual information) shown in the image. (3) Select appropriate information from the plurality of visual information obtained according to the judgment algorithm implemented by the smart glass 100 and distribute (broadcast) it to other surrounding smart glasses 100 (the second smart glass 100b). (4) The position information of the smart glass 100 may be distributed together with the visual information. (5) Other smart glasses 100 that have received visual information determine whether to use the received information according to the judgment criteria of the application used by the own device. (6) Other smart glasses 100 (second smart glasses 100b) generate and output at least one of voice, image (video), and AR model from the received visual information (attention notification). (7) Other smart glasses 100 that have received visual information determine whether to perform redistribution according to the implemented redistribution algorithm. (8) Processes that require resource capabilities such as image processing and AR rendering may be offloaded to mobile terminals (computing devices) (computers) such as smartphones and tablets connected to the smart glasses 100 (other smart glasses 100). The smart glasses system 1 adopts a mesh method using a plurality of smart glasses 100, enabling judgment processing with partial information, not requiring other external equipment (other infrastructure) such as a server, having no location restrictions, not requiring a large amount of equipment investment (equipment cost), and being able to reduce the communication load because, for example, it is not necessary to use a WAN (Wide Area Network) line (wide area communication network).

[0011] That is, as an example, the smart glasses system 1 can extract visual information by the camera 121 and the AI-equipped smart glasses 100 and share the visual information with unspecified third parties (other smart glasses 100) around. As a more specific example, when one smart glass 100 (the first smart glass 100a) generates image information by imaging with a camera 121 mounted on the own device and estimates that a "traffic accident" has occurred in response to the analysis and processing of the image information by a control unit 110 (local AI) mounted on the own device, it generates visual information indicating that a traffic accident has occurred and transmits the visual information to another smart glass 100 (the second smart glass 100b) via a communication unit 122. When the other smart glass 100 receives the visual information via the communication unit 122, it outputs an attention notification by AR or the like. Note that the attention notification is not limited to the attention notification by AR, and may be an attention notification by an image (still image and moving image) and sound (voice) or the like.

[0012] Such a smart glass 100 may be, for example, a visual information sharing system using a smart glass mesh composed of a plurality of smart glasses 100. Since the smart glass system 1 shares visual information (transmits it from one smart glass 100 to another smart glass 100) among a plurality of smart glasses 100 instead of image information, for example, it is possible to consider the privacy of a person or the like recorded in the image information, and since it transmits visual information with a smaller data amount than the image information, the communication volume can be reduced.

[0013] Note that "smart glass" can be equivalently referred to as a "computer device" such as a "wearable terminal" and a "wearable device". That is, the smart glass 100 may be, for example, a glasses-type computer device or the like. Therefore, the "smart glass system" may be equivalently referred to as a "computer device system" such as a "wearable terminal system" and a "wearable device system". That is, the smart glass system 1 may be, for example, a computer system including a plurality of glasses-type computer devices or the like.

[0014] [Details of Smart Glass 100] Next, the smart related to one embodiment will be described in detail. FIG. 2 is a block diagram for explaining a smart glass 100 according to one embodiment.

[0015] Each of the plurality of smart glasses 100 includes, for example, a camera 121, a communication unit 122, a storage unit 123, a display unit 124, a control unit 110, and the like. The control unit 110 may be configured by, for example, an arithmetic processing unit of an information processing device. The control unit 110 (for example, an arithmetic processing unit, etc.) may realize various processing functions by appropriately reading and executing various programs and the like stored in the storage unit 123 and the like. That is, the processing functions may be realized by computer implementation.

[0016] The camera 121 (camera unit) images, for example, the field of view (at least a part of the field of view) of the user wearing the smart glass 100 and generates image information.

[0017] The communication unit 122 is, for example, a communication interface capable of transmitting and receiving various information to and from a device (external device) outside the information processing device. The communication unit 122 can communicate with, for example, one or a plurality of smart glasses 100. In addition, the communication unit 122 may be capable of receiving positioning satellite signals using a positioning satellite system such as GPS and GNSS. Thereby, the smart glass 100 can acquire position information and the like.

[0018] The storage unit 123 may store, for example, various information and programs. An example of the storage unit 123 may be a memory, a solid state drive, a hard disk drive, and the like.

[0019] The display unit 124 is, for example, a display capable of displaying various characters, symbols, images, and the like. The display unit 124 may be, for example, the lens surface of the smart glass 100 and the like.

[0020] Note that the smart glasses 100 may be connected to a mobile terminal (not shown) such as a smartphone and a tablet, and a part of the processing functions of the control unit 110 may be realized by the control unit (terminal control unit) (not shown) of the mobile terminal (offload). In this case, the terminal control unit of the mobile terminal (for example, an arithmetic processing unit, etc.) may realize various processing functions including, for example, an image recognition (analysis processing) function based on the image information generated by the camera 121, an AR and image (still image and moving image) generation function for attention notification, and a communication function with other smart glasses 100 (mobile terminal connected to other smart glasses 100).

[0021] When a plurality of the above-described smart glasses 100 are gathered, the smart glass system 1 is configured. The plurality of smart glasses 100 can communicate with each other by using their own communication unit 122.

[0022] The smart glass system 1 includes a plurality of smart glasses 100.

[0023] (Details of the first smart glasses 100a) Next, the details of the first smart glasses 100a among the plurality of smart glasses 100 will be described. FIG. 3 is a block diagram for explaining the first smart glasses 100a and the second smart glasses 100b.

[0024] The first smart glasses 100a are smart glasses that serve as a transmission source of visual information.

[0025] As shown by an example in FIG. 3, the first smart glasses 100a among the plurality of smart glasses 100 include a camera 121, a communication unit 122 (first communication unit 122a), a storage unit 123, a display unit 124, and a control unit 110 (first control unit 110a), etc.

[0026] The camera 121 captures an image in the direction including the field of view of the user of the first smart glass 100a to generate image information. The camera 121 may, for example, capture a video or capture a still image at predetermined intervals. The camera 121 may, for example, capture an image in front of the user.

[0027] The first control unit 110a performs, for example, image recognition on the image information generated by the camera 121 and extracts the objects and events recorded in the image information. The first control unit 110a may perform, for example, image analysis based on the image information and acquire the content of the objects recorded in the image information. Here, an example of the objects extracted by the first control unit 110a may be a person, a bicycle, a motorcycle, an automobile, an electric kick scooter, etc. That is, the content of the object may be, for example, the type of the object (e.g., a person, a bicycle, a motorcycle, an automobile, etc.), whether the objects are the same or different, whether the object is moving or not, if the object is moving, the moving direction and speed of the object, the distance to the object, etc. Also, an example of the events extracted by the first control unit 110a may be the occurrence of an accident, a dangerous driving vehicle, a pedestrian to be considered (such as an elderly person), traffic jam - density of people, etc. That is, the content of the event may be, for example, the occurrence of an accident, a dangerous driving vehicle, a pedestrian to be considered (such as an elderly person), traffic jam - density of people, etc.

[0028] The first communication unit 122a transmits the visual information recording the content of the objects and events extracted by the first control unit 110a to one or more second smart glasses 100b different from the first smart glass 100a among the plurality of smart glasses 100. In this case, the first communication unit 122a may broadcast the visual information to one or more second smart glasses 100b existing around the first smart glass 100a. That is, the first communication unit 122a may transmit the visual information to one or more second smart glasses 100b (unspecified second smart glasses 100b). Here, "broadcast" means, for example, transmitting to all second smart glasses 100b existing within a predetermined range (the communication range of the first communication unit 122a).

[0029] (Determination of Priority) Next, the determination (setting) of the priority when transmitting (distributing) visual information will be described.

[0030] (First Example of Priority Determination) First, the first example of priority determination will be described.

[0031] The first control unit 110a may extract one or more objects and events continuously in time based on, for example, image information. When the objects continuously extracted in time are the same object, the first control unit 110a may acquire the content of the same object including the moving direction of the same object and the distance from the first smart glass 100a to the same object as the moving characteristics. The moving characteristics (content of the object (same object)) may be, for example, as described above, the type of the object (person, bicycle, motorcycle, automobile, etc.), whether the objects are the same or different, whether the object is moving or not, and if the object is moving, the moving direction and speed of the object, the distance to the object, etc.

[0032] In this case, the first control unit 110a may determine the priority when transmitting the visual information of the same object to the second smart glass 100b according to, for example, the moving characteristics. As an example, when the moving speed of the same object is relatively fast, when the distance from the first smart glass 100a to the same object tends to become shorter in time (when the same object approaches the first smart glass 100a), and when the distance between the first smart glass 100a and the same object is relatively short, the first control unit 110a may group them and determine that the priority is relatively high if it corresponds to at least one of the cases in the group. Also, when the first control unit 110a corresponds to a plurality of cases in the group, it may determine that the priority is higher than when it corresponds to one case.

[0033] The first control unit 110a may include the content and movement characteristics of the object extracted (acquired) as described above in the visual information. The visual information may be, for example, the type of the object (person, bicycle, motorcycle, automobile, etc.), whether the objects are the same or different, whether the object is moving or not, if the object is moving, the moving direction and moving speed of the object, the distance to the object, etc.

[0034] For example, the first control unit 110a can acquire position information (first position information) indicating the position of the first smart glasses 100a (own device) by using the positioning satellite signals acquired by the communication unit 122. For example, the first control unit 110a may control the first communication unit 122a to include the first position information indicating the position of the first smart glasses 100a (own device) in the visual information and transmit the visual information to the second smart glasses 100b according to the priority. As an example, the first control unit 110a may control the first communication unit 122a to transmit the visual information (for example, including the first position information) of the object (same object) determined to have a relatively high priority to one or more of the second smart glasses 100b described later. As another example, for the visual information of the object (same object) determined to have a relatively low priority, the first control unit 110a may not transmit it to the second smart glasses 100b, or may control the first communication unit 122a to transmit it to one or more of the second smart glasses 100b.

[0035] (Second example of priority determination) Next, a second example of priority determination will be described.

[0036] As described above, the first control unit 110a may extract one or more objects continuously in time based on the image information. In this case, the first control unit 110a may acquire, as the movement characteristics, the content of the object including, for example, the moving direction of each of the one or more objects continuously extracted in time and the distance from the first smart glasses 100a to each of the one or more objects.

[0037] In this case, for example, when extracting a plurality of objects, the first control unit 110a may determine that at least one of the distance from the first smart glass 100a, the moving direction of the object, and the moving speed of the object has a relatively high priority according to the moving characteristics of each of the plurality of objects. As a specific example, when extracting a plurality of objects, the first control unit 110a may use the moving characteristics of each of the plurality of objects to determine that at least one of the object closer to the first smart glass 100a, the object approaching the first smart glass 100a, and the object with a higher moving speed has a relatively high priority. That is, as an example, when the moving speed of the object is relatively high, when the distance from the first smart glass 100a to the object tends to become shorter over time (when the same object approaches the first smart glass 100a), and when the distance between the first smart glass 100a and the object is relatively short, the first control unit 110a may group these cases, and if it corresponds to at least one case in the group, it may determine that the priority is relatively higher than when it does not correspond to each case.

[0038] The first control unit 110a may control the first communication unit 122a to transmit the visual information of the object with a relatively high priority to the second smart glass 100b. As an example, the first control unit 110a may control the first communication unit 122a to transmit the visual information (for example, including the first position information) of the object determined to have a relatively high priority to one or more of the second smart glasses 100b described below. As another example, for the visual information of the object determined to have a relatively low priority, the first control unit 110a may not necessarily transmit it to the second smart glass 100b, or may control the first communication unit 122a to transmit it to one or more second smart glasses 100b.

[0039] (The third example of priority determination) Next, the third example of priority determination will be described.

[0040] The first control unit 110a may obtain, for example, the second position information and the movement direction information of the second smart glasses 100b from the second smart glasses 100b via the communication unit 122. The second position information may be, for example, information (position information) recording the position obtained by the second smart glasses 100b. The movement direction information may be, for example, information indicating the movement direction of the second smart glasses 100b based on the movement trajectory of the position obtained by the second smart glasses 100b.

[0041] In this case, the first control unit 110a may determine the priority of the object based on, for example, the movement characteristics of the object obtained as described above and the position and movement direction of the second smart glasses 100b (the position of the second smart glasses 100b recorded in the second position information and the movement direction recorded in the movement direction information). When the first control unit 110a determines the priority of the object as described above, the first control unit 110a may control the first communication unit 122a to transmit the visual information of the object with a relatively high priority to the second smart glasses 100b.

[0042] As an example in this case, the first control unit may identify an object approaching the second smart glasses 100b based on the movement characteristics of the object and the position and movement direction of the second smart glasses 100b (the position of the second smart glasses 100b recorded in the second position information and the movement direction recorded in the movement direction information). The first control unit 110a may estimate the position of the object based on the first position information obtained by the first smart glasses 100a, and determine whether the object approaches the second smart glasses 100b based on the estimated position of the object, the movement characteristics of the object, and the position and movement direction of the second smart glasses 100b. For example, when the first control unit 110a determines that an object is approaching the second smart glass 100b, it may determine that the priority of the object approaching the second smart glass 100b is relatively high. On the other hand, for example, when the first control unit 110a determines that an object is moving away from the second smart glass 100b, it may determine that the priority of the object approaching the second smart glass 100b is relatively low. That is, the first control unit 110a may set the priority of an object approaching the second smart glass 100b higher than the priority of an object moving away from the second smart glass 100b. When the first control unit 110a sets the priority of an object as described above, the first communication unit 122a may be controlled to transmit the visual information of the object with a relatively high priority to the second smart glass 100b.

[0043] (Details of the second smart glass 100b) Next, the details of the second smart glass 100b among the plurality of smart glasses 100 will be described. The second smart glass 100b is a smart glass that serves as a receiver of visual information, or a source or destination of transfer of visual information.

[0044] As illustrated in FIG. 3, the second smart glass 100b among the plurality of smart glasses 100 includes a camera 121, a communication unit 122 (second communication unit 122b), a storage unit 123, a display unit 124, a control unit 110 (second control unit 110b), and the like.

[0045] The second communication unit 122b receives the visual information transmitted from the first communication unit 122a of the first smart glass 100a.

[0046] The second control unit 110b generates predetermined content about the object and the event according to the content of the object recorded in the visual information received by the second communication unit 122b, and controls to display the predetermined content on the second smart glass 100b. That is, the second control unit 110b may determine, for example, whether the object is an object that should be noted based on the visual information (the content of the object recorded in the visual information) and the determination criteria preset in the own device. When the second control unit 110b determines that the object recorded in the visual information is an object that should be noted, it generates content indicating attention as the predetermined content about the object and the event. In this case, the second control unit 110b may generate, for example, the predetermined content (such as content indicating attention, etc.) about the object and the event by a generation AI. The content indicating attention may be various contents such as the existence of the object that should be noted, the object that should be noted approaching, the moving direction and distance of the object that should be noted, etc., by AR, voice (sound), and image (still image and moving image), etc. Note that the predetermined content about the object is not limited to the content indicating attention to the object as described above, but may also be content for conveying to the user the object and the event extracted by the first control unit 110a. As a specific example, it may be various contents including attention, the presence or absence of litter, information on a desired store, etc.

[0047] The second control unit 110b outputs the content indicating attention from the second smart glass 100b. As an example, in the case of the content indicating attention by AR and image, etc., the second control unit 110b displays it on the lens surface (display unit 124) of the second smart glass 100b. Also, in the case of the content indicating attention by voice (sound), for example, the second control unit 110b outputs it from a speaker (not shown) arranged in the second smart glass 100b.

[0048] The second control unit 110b can acquire position information (second position information) indicating the position of the second smart glass 100b (own device), for example, by using the positioning satellite signal acquired by the second communication unit 122b. When the second control unit 110b receives, for example, the visual information transmitted by the first communication unit 122a, it may determine whether the same object is an object that should be noted based on the movement characteristics and the first position information recorded in the visual information, and the second position information indicating the position of the own device acquired by the second smart glass 100b. For example, the second control unit 110b may make various determinations including whether the object (the same object) is approaching the second smart glass 100b (the own device), whether the position of the object (the same object) and the second smart glass 100b (the own device) are relatively close, whether the movement speed of the object (the same object) is relatively fast, etc. If they match, it may be determined that the object (the same object) is an object that should be noted. The above-mentioned matching cases may be, for example, the case where the object (the same object) is approaching the second smart glass 100b (the own device), the case where the position of the object (the same object) and the second smart glass 100b (the own device) are relatively close, the case where the movement speed of the object (the same object) is relatively fast, etc. In this case, when the second control unit 110b determines whether the object (the same object) is an object that should be noted, for example, similar to the above-mentioned cases, as predetermined content about the object and the event, it may generate content indicating attention to the same object and control to display that content indicating attention.

[0049] FIG. 4 is a first diagram for explaining an example of the determination criteria of the second smart glass 100b.

[0050] As illustrated in FIG. 4, when the first smart glass 100a recognizes two bicycles (Bicycle A and Bicycle B), it generates visual information A about Bicycle A and visual information B about Bicycle B, and distributes both visual information A and visual information B to a plurality of second smart glasses 100b existing around. The visual information A and the visual information B may record the result of the first smart glass 100a recognizing an object (bicycle), whether each of the two objects (bicycles) is approaching or moving away from the first smart glass 100a, the distance between each of the two objects (bicycles) and the first smart glass 100a, the position information (first position information) of the first smart glass 100a, and the information about the moving direction of the first smart glass 100a respectively.

[0051] As illustrated in FIG. 4, each of the plurality of second smart glasses 100b receives the visual information A and the visual information B. The second smart glass 100b determines whether Bicycle A and Bicycle B should be noted based on various information about its own device (for example, the position information (second position information) of its own device, the moving direction of its own device, the moving speed of its own device, and a map (map information), etc.) and the information recorded in each of the visual information A and the visual information B. For example, since one of the second smart glasses 100b approaches Bicycle A, it determines that Bicycle A (visual information A) is an object to be noted. Similarly, for example, since one of the second smart glasses 100b moves away from Bicycle B, it determines that Bicycle B (visual information B) is an object that does not need to be noted. For example, since the other second smart glass 100b moves away from Bicycle A and Bicycle B respectively, it determines that Bicycle A (visual information A) and Bicycle B (visual information B) are objects that do not need to be noted.

[0052] FIG. 5 is a second diagram for explaining an example of the determination criteria of the second smart glass 100b. FIG. 5(A) is a diagram for explaining the transmission of visual information from the first smart glass 100a to the second smart glass 100b and the feedback on the use of the visual information in the second smart glass 100b. FIG. 5(B) is a diagram for explaining the transmission of visual information from the first smart glass 100a to the second smart glass 100b after the process of FIG. 5(A).

[0053] As illustrated in FIG. 5(A), the first smart glass 100a distributes the visual information C and the visual information D to the second smart glass 100b in the same manner as illustrated in FIG. 4. The visual information C is the visual information of the bicycle C, and the visual information D is the visual information of the bicycle D. As illustrated in FIG. 5(A), after receiving the visual information C and the visual information D, the second smart glass 100b determines that the bicycle D approaches the own device based on the visual information C and the visual information D, and determines that the bicycle D should be noted. The second smart glass 100b transmits (feeds back to the first smart glass 100a) the information of the determination result that the bicycle D should be noted (indicating that only the visual information D out of the visual information C and the visual information D is used) to the first smart glass 100a. As illustrated in FIG. 5(B), based on the feedback from the second smart glass 100b (the second smart glass 100b determines that the bicycle D should be noted (indicating that only the visual information D out of the visual information C and the visual information D is used)), the first smart glass 100a transmits the visual information D out of the visual information C and the visual information D to the second smart glass 100b.

[0054] (Redistribution of Visual Information) Next, an example in which the second smart glass 100b transmits (redistributes) visual information to another second smart glass 100b will be described. FIG. 6 is a first diagram for explaining an example of redistribution (transfer).

[0055] As illustrated in FIG. 6, for example, when the second control unit 110b receives visual information from the first smart glass 100a, it may re - distribute (transmit (transfer)) the visual information to other devices (other second smart glasses 100b) existing around the own device (the second smart glass 100b). That is, for example, when the second control unit 110b receives visual information transmitted (distributed) from the first smart glass 100a, it may control the second communication unit 122b to transmit (re - distribute) the visual information to one or a plurality of other second smart glasses 100b existing around the second smart glass 100b. Similarly, the second control unit 110b of other devices (other second smart glasses 100b) may repeatedly transmit (re - distribute) the visual information to even other devices (even other second smart glasses 100b) existing around the other devices (other second smart glasses 100b) when it receives the visual information, for example.

[0056] (First example of re - distribution) Next, the first example of the re - distribution described above will be explained.

[0057] The second control unit 110b of a plurality of second smart glasses 100b, that is, the second control unit 110b of each of the own device (the second smart glass 100b) and other devices (other second smart glasses 100b) (including even other devices (even other second smart glasses 100b)) may record the number of transmissions in the visual information every time the same visual information is transmitted. That is, the second control unit 110b may count up the number of transmissions recorded in the visual information every time the visual information is transmitted. That is, the second control unit 110b of each of the plurality of second smart glasses 100b may count up the number of transmissions by 1 each time it transmits (re - distributes) the visual information, record the number of transmissions in the visual information, and transmit the visual information in which the counted - up number of transmissions is recorded. The second control unit 110b may be able to repeat transmitting visual information to another second smart glass 100b different from its own device until the number of transmissions reaches a preset threshold value (first threshold value). That is, as an example, when the number of transmissions that becomes the first threshold value is 5 times, the second control unit 110b can transmit visual information to another device (another second smart glass 100b) until the number of transmissions reaches 5 times, and it may be made impossible to transmit visual information to another device (another second smart glass 100b) when the number of transmissions is 6 times or more.

[0058] (Second example of redistribution) Next, a second example of redistribution will be described.

[0059] Each second control unit 110b of the plurality of second smart glasses 100b, that is, the second control unit 110b of its own device (second smart glass 100b) and other devices (other second smart glasses 100b) (including further other devices (further other second smart glasses 100b)), may record the distance over which the visual information was transmitted in the visual information each time the visual information is received. In this case, the second control unit 110b of its own device (second smart glass 100b) may record the position of its own device (second position information) in the visual information and transmit it to other devices (other second smart glasses 100b) when transmitting the visual information. When the second control unit 110b of another device (another second smart glass 100b) receives the visual information, it may obtain the distance between the position of the other device and the position of its own device based on the position of the other device (second position information) and the position of its own device (second position information) recorded in the visual information. The second control unit 110b of another device (another second smart glass 100b) may, for example, use the obtained distance as the transmission distance of the visual information. The second control unit 110b of another device (another second smart glass 100b) may, for example, record the obtained transmission distance in the visual information. That is, each second control unit 110b of the own device (the second smart glass 100b) and other devices (other second smart glasses 100b) may record, for example, the transmission distance at which the visual information is transmitted in the visual information when receiving the visual information. In other words, each second control unit 110b of the plurality of second smart glasses 100b may record the second position information in the visual information each time the same visual information is transmitted, and acquire the transmission distance each time the same visual information is received.

[0060] Furthermore, each second control unit 110b of the plurality of second smart glasses 100b may acquire the total transmission distance (total transmission distance) obtained by summing up the transmissions (redistributions) over multiple times when the same visual information is transmitted (redistributed) multiple times. Each second control unit 110b of the plurality of second smart glasses 100b may stop transmitting (redistributing) the same visual information to other second smart glasses 100b different from the own device when the transmission distance of the same visual information (total transmission distance which is the sum of one or more transmission distances) exceeds a preset threshold value (second threshold value).

[0061] (Third Example of Redistribution) Next, the third example of redistribution will be described.

[0062] The second control unit 110b of the own device (the second smart glasses 100b) may acquire the position (second position information) of another device (another second smart glasses 100b) and the moving direction (moving direction information) of the other device from the other device. In this case, the second control unit 110b of the own device may acquire the second position information and the moving direction information from the other device, or may acquire the second position information and the moving direction information from the visual information (including the second position information and the moving direction information) distributed by the other device when receiving the visual information. That is, the second control unit 110b of the own device (the second smart glasses 100b) may acquire the second position information about the position acquired by the other device (another second smart glasses 100b) and the moving direction information indicating the moving direction based on the moving trajectory of the position from the other device (another second smart glasses 100b). The second control unit 110b of the own device (the second smart glasses 100b) may determine whether the object is an object that should be noted for the other device (another second smart glasses 100b) based on the second position information of the other device, the moving direction information of the other device, and the moving characteristics of the object recorded in the visual information. As an example, the second control unit 110b of the own device (the second smart glasses 100b) may perform various determinations such as whether the object is approaching another device (another second smart glasses 100b), whether the distance between the object and another device (another second smart glasses 100b) is relatively short, whether the moving speed of the object is relatively fast, etc., and may determine that the object is an object that should be noted when the conditions are met. The above-mentioned case where the conditions are met may be, for example, when the object is approaching another device (another second smart glasses 100b), when the position of the object and another device (another second smart glasses 100b) is relatively close, when the moving speed of the object is relatively fast, etc. In this case, when the second control unit 110b of the own device (the second smart glasses 100b) determines that the object is an object that should be noted for the other device (another second smart glasses 100b), it may re-distribute (transmit) the visual information to the other device (another second smart glasses 100b).

[0063] (Fourth Example of Re-Distribution) Next, the fourth example of re-distribution will be described. FIG. 7 is a second diagram for explaining an example of re - distribution (transfer).

[0064] As illustrated in FIG. 7, when there are a plurality of first smart glasses 100a (first smart glasses E, H) that have detected the same object, the second smart glass 100b (second smart glass F) can acquire visual information from each of the plurality of first smart glasses 100a (first smart glasses E, H). Here, the second control unit 110b of the second smart glass 100b (second smart glass F) may determine whether the objects (a bicycle in the case illustrated in FIG. 7) recorded in each visual information are the same or not based on the position information recorded in each of the plurality of visual information. When the objects recorded in each visual information are the same, the second control unit 110b of the second smart glass 100b (second smart glass F) may transmit (re - distribute (transfer)) one of the plurality of visual information to another second smart glass 100b (second smart glass G).

[0065] (Grouping of Smart Glasses 100) Next, the grouping of the smart glasses 100 will be described.

[0066] Each of the control units 110 (first control unit 110a and second control unit 110b) of the plurality of smart glasses 100 (first smart glasses 100a and second smart glasses 100b) may acquire the position information of each device and the moving direction information of each device. That is, the control unit 110 may acquire position information (first position information and second position information) indicating the position of the own device (each device) acquired by each device (each smart glass 100) and moving direction information indicating the moving direction of each device (the moving direction of each device) indicated by the locus of that position. The control unit 110 identifies the smart glasses 100 existing within a predetermined distance range based on the position information of each device. That is, the control unit 110 determines whether a plurality of smart glasses 100 exist in the surroundings based on the position information of each device. When the control unit 110 determines that a plurality of smart glasses 100 exist in the surroundings, that is, when the smart glasses 100 existing within the predetermined distance range are identified, the control unit 110 sets and groups the plurality of smart glasses 100 into one group. Further, the control unit 110 determines a representative of the group from within the group. In this case, for example, the control unit 110 may determine, as the representative of the group, the smart glass 100 (first smart glass 100a) that is closest to an arbitrary object to be noted (for example, the object with the highest priority set as described above) among the set group. That is, the control unit 110 may set the smart glass 100 serving as the representative of the group as the first smart glass 100a. The first control unit 110a of the first smart glass 100a may regard other smart glasses 100 for which grouping has not been performed as the second smart glass 100b. That is, the first control unit 110a sets other smart glasses 100 outside the group as the second smart glass 100b. The first control unit 110a may transmit the visual information generated by the first smart glass 100a to the second smart glass 100b.

[0067] [Information Processing Method] Next, the outline of the information processing method according to an embodiment will be described. FIG. 8 is a flowchart for explaining the outline of the information processing method according to an embodiment.

[0068] In step ST101, the camera 121 of the first smart glass 100a images the direction including the field of view of the user of the first smart glass 100a and generates image information.

[0069] In step ST102, the first control unit 110a of the first smart glass 100a performs image recognition on the image information generated in step ST101, and extracts the objects and events recorded in the image information. The first control unit 110a may acquire the movement characteristics of the object, and set the priority when transmitting visual information in step ST103 described later according to the movement characteristics. The first control unit 110a acquires position information (second position information) and a movement direction (movement direction information) from the second smart glass 100b described later, and sets the priority when transmitting visual information in step ST103 according to the position of the second smart glass 100b and the movement direction of the second smart glass 100b. After the first control unit 110a transmits the visual information to the second smart glass 100b in step ST103 described later, the first control unit 110a may set the priority according to the feedback from the second smart glass 100b.

[0070] In step ST103, the first communication unit 122a of the first smart glass 100a transmits (distributes) the visual information recording the content of the object extracted in step ST102 to the second smart glass 100b. In this case, the first communication unit 122a may broadcast the visual information to one or a plurality of second smart glasses 100b existing around the first smart glass 100a. The first communication unit 122a may transmit the visual information of the object whose priority is set relatively high in step ST102 to the second smart glass 100b. The first communication unit 122a may transmit the visual information including the first position information indicating the position of the first smart glass 100a.

[0071] In step ST104, the second communication unit 122b of the second smart glass 100b receives the visual information transmitted in step ST103.

[0072] In step ST105, the second control unit 110b of the second smart glass 100b generates predetermined content about the object according to the content of the object recorded in the visual information received in step ST104, and controls to output the predetermined content. The second control unit 110b may determine whether the object recorded in the visual information is an object to be noted, and if it is determined to be an object to be noted, control to output a notice indicating that the object should be noted. The second control unit 110b of the second smart glass 100b may transmit (redistribute) (transfer) the visual information to another second smart glass 100b different from its own device. The plurality of second smart glasses 100b may repeat the redistribution until the total number of repetitions of the redistribution reaches a threshold value (the first threshold value). Also, the plurality of second smart glasses 100b may repeat the redistribution until the total power transmission distance of the repeated redistribution reaches a threshold value (the second threshold value). Also, when the object is an object to be noted for another second smart glass 100b, the second control unit 110b of the second smart glass 100b may transmit (redistribute) the visual information of the object.

[0073] Note that when the above-described first smart glass 100a and second smart glass 100b (smart glass 100) exist around each other (for example, when they exist within a preset predetermined distance range), a plurality of smart glasses 100 within that range may be grouped into one group. One smart glass 100 may be set as a representative in one group, and the smart glass 100 set as the representative may mainly transmit (distribute) the visual information.

[0074] [Regarding the program] The control unit 110 of the smart glass 100 may be realized as a function of a computing processing device of a computer or the like. The smart glass 100 may, for example, realize the processing function by one control unit 110 (for example, a computing processing device or the like), or may distribute and realize the processing function by a plurality of different control units 110 (for example, a computing processing device or the like). The information processing program can implement processing functions on the smart glasses 100 (computer). The information processing program may be recorded on a non-transitory tangible recording medium readable by a computer, such as a memory, a solid state drive, a hard disk drive, or an optical disk. The storage medium may be, for example, rephrased as a non-transitory tangible computer-readable medium that stores the information processing program. Also, the information processing program may be transmitted online. The information processing program can be realized by the control unit 110 (for example, an arithmetic processing unit, etc.) as a product (computer program product) including the information processing program.

[0075] The smart glasses 100 can combine one or any plurality of the above-described processes. In the present disclosure, the term "information" is used, but the term "information" can be rephrased as "data", and the term "data" can be rephrased as "information".

[0076] [Aspects of the present embodiment, effects of each aspect, and examples of each aspect] Next, one aspect of the present embodiment and the effects of each aspect will be described. Note that each aspect described below is an example at the time of filing, and the present embodiment is not limited to the aspects described below. That is, the present embodiment is not limited to the aspects described below, and may be realized by appropriately combining the above-described respective parts. Also, lower-level aspects may be cited by any of the higher-level aspects. Also, the effects of the present embodiment described below are examples, and the effects of each aspect are not limited to those described below. Also, each aspect may exhibit, for example, at least one of the effects described below.

[0077] (Aspect 1) One aspect of the smart glasses system includes a plurality of smart glasses capable of communicating with each other, and each of the plurality of smart glasses includes a camera, a communication unit, and a control unit. Among a plurality of smart glasses, a first smart glass includes a camera that captures an image in a direction including the field of view of the user of the first smart glass to generate image information, a first control unit as a control unit that performs image recognition on the image information generated by the camera and extracts objects and events recorded in the image information, and a first communication unit as a communication unit that transmits visual information recording the contents of the objects and events extracted by the first control unit to one or a plurality of second smart glasses different from the first smart glass among the plurality of smart glasses. The second smart glass includes a second communication unit as a communication unit that receives the visual information transmitted from the first communication unit of the first smart glass, and a second control unit as a control unit that generates predetermined contents about the object and the event according to the content of the object recorded in the visual information received by the second communication unit and controls to display the predetermined contents on the second smart glass. Thereby, the smart glass system can share visual information such as objects and events acquired by one smart glass (the first smart glass) with other smart glasses (the second smart glass). In the smart glass system, for example, as the content to be shared, the occurrence of an accident, a dangerous driving vehicle, a pedestrian to be considered (such as an elderly person), and traffic congestion and population density can be mentioned. The smart glass system can also be used for the safe operation support of not only pedestrians but also two-wheeled vehicles and electric kick scooters. The smart glass system can notify the user of the second smart glass of the predetermined content as the display of the predetermined content.

[0078] Here, as an example, when citing the use example of the smart glass system as the sharing of traffic information, the smart glass system may perform the following processes (1) to (8). (1) The first smart glass captures an image with a camera mounted on the own device. (2) The first smart glass analyzes the image in (1) above and detects traffic moving objects such as pedestrians, bicycles, two-wheeled vehicles, and automobiles. (3) The first smart glass determines the movement characteristics of the detected traffic moving objects (such as the presence or absence of movement, movement direction, distance, etc.) by continuously performing the analysis in (2) above. (4) The first smart glass determines the distribution priority of the visual information to be distributed in (5) described later according to the movement characteristics in (3) above. (5) The first smart glass distributes (broadcasts) visual information (such as information on traffic moving objects, information on movement characteristics, information on the position of its own device, information on the movement direction of its own device, etc.) to other smart glasses in the vicinity according to the distribution priority in (4) above. (6) The second smart glass (information receiver) determines whether to pay attention to the traffic moving objects detected by the first smart glass based on visual information, the position information of its own device, the movement direction of its own device, the movement speed of its own device, and map information, etc. (7) When the second smart glass determines in (6) above that the traffic moving object should be paid attention to, it generates and outputs a voice, an image (video or still image), or AR indicating that the traffic moving object is an object that should be paid attention to based on visual information, etc. (8) The second smart glass determines whether to re - distribute the visual information to another second smart glass according to its own re - distribution algorithm, and if it determines to re - distribute, it re - distributes (re - broadcasts) the visual information. In (6) above, the second smart glass may determine that the traffic moving object detected by the first smart glass should be paid attention to when the own device and the traffic moving object approach based on the movement direction of the own device and the movement direction of the traffic moving object. On the other hand, the second smart glass may determine that there is no need to pay attention to the traffic moving object detected by the first smart glass when the own device and the traffic moving object move away from each other based on the movement direction of the own device and the movement direction of the traffic moving object. Similarly, in (6) above, the second smart glass may determine that there is no need to pay attention to the traffic moving object detected by the first smart glass when the movement direction of the own device moves away from the movement direction of the traffic moving object.

[0079] In addition, in a smart glass system, the smart glass that distributes visual information can perform distribution with location information attached. In this case, the smart glass may identify the intersection that is relatively close (for example, the closest) to the smart glass by referring to its own position (location information) and the map information. The intersections recorded in the map information may be given identification information for identifying intersections and the like. The identification information may be, for example, identification information represented by 16 bits or the like. Also, the moving direction of an object may be represented by 3 bits or the like by representing it in each direction of north, northeast, east, southeast, south, southwest, west, and northwest (representing the general direction). The smart glass system can represent the position and the moving direction in a total of 19 bits, for example, by representing the position of the smart glass and the position of the object in terms of the position of the intersection (identification information (16 bits)) and representing the moving direction of the object in terms of the general direction (3 bits). When represented in terms of longitude and latitude coordinates, 32 bits are required for each of longitude and latitude. Thereby, the smart glass system can reduce the amount of information such as visual information to be distributed.

[0080] Also, in the smart glass system, the smart glass may use generative AI to detect abnormal states, generate text, generate images, and generate AR images based on visual information. As an example, the first smart glass may use generative AI to detect abnormal states such as "a vehicle traveling at high speed," "a vehicle that has overturned," and "a collision between vehicles" as the content of an object based on the result of recognizing image information, and generate text explaining the abnormal state. The first smart glass may include the generated text in the visual information and distribute it. As another example, the second smart glass may use generative AI to generate images and AR images from the text recorded in the visual information, and use the images and AR images to visually notify the user of an abnormality (attention).

[0081] (Aspect 2) In a smart glass system according to one aspect, the first communication unit of the first smart glass may broadcast visual information to one or more second smart glasses existing around the first smart glass. Thereby, the smart glass system can share visual information such as objects and events acquired by one smart glass (the first smart glass) with other smart glasses (the second smart glasses).

[0082] (Aspect 3) In a smart glass system according to one aspect, the first control unit of the first smart glass continuously extracts objects and events over time, and obtains movement characteristics as the content of the same object, including the movement direction of the same object among the extracted objects and the distance from the first smart glass to the same object. According to the movement characteristics, determine the priority when transmitting the visual information of the same object to the second smart glass, include the first position information indicating the position of the own device acquired by the first smart glass in the visual information, and control the first communication unit to transmit the visual information to the second smart glass according to the priority. Thereby, the smart glass system can set the priority according to, for example, the movement characteristics of the object. As a specific example, the first smart glass may increase the priority in the order of when the object is a motorcycle > a bicycle > a pedestrian, with the former having a higher priority. Also, as a specific example, the first smart glass may increase the priority in the order of when the object is approaching the own device > moving away from the own device > stopped, with the former having a higher priority. Also, as a specific example, the first smart glass may increase the priority in the order of when the distance between the object and the own device is relatively close > relatively far, with the former having a higher priority. The first smart glass can set the priority according to, for example, one of the above three specific examples of movement characteristics, or a combination of a plurality of the above three specific examples of movement characteristics. Further, the first smart glass can obtain the position and moving direction of the second smart glass, for example, from the second smart glass to which visual information is to be transmitted (receive feedback from the second smart glass), and set a priority according to the position and moving direction of the second smart glass (feedback from the second smart glass). Also, when the first smart glass obtains, for example, from the second smart glass to which visual information is to be transmitted, the result (by the second smart glass broadcasting the result) of outputting (already output) attention based on the visual information in the second smart glass (receiving feedback from the second smart glass), the priority for the visual information (the object corresponding to the visual information) may be set higher.

[0083] (Aspect 4) In a smart glass system according to an aspect, when the second control unit of the second smart glass receives visual information transmitted by the first communication unit, based on the movement characteristics and first position information of the same object recorded in the visual information and the second position information indicating the position of the own device acquired by the second smart glass, it determines whether the same object is an object that should be paid attention to. When determining whether the same object is an object that should be paid attention to, as predetermined content about the object and the event, it may generate content indicating attention to the same object and control to display the content indicating attention. Thereby, the smart glass system shares visual information such as objects and events obtained by one smart glass (the first smart glass) with other smart glasses (the second smart glass), and the other smart glasses (the second smart glass) can present (output) attention content to the user of the own device based on the visual information.

[0084] (Aspect 5) In a smart glass system according to one aspect, when the first control unit of the first smart glass extracts a plurality of objects, it uses the movement characteristics of each of the plurality of objects to determine that the priority is relatively high according to at least one of the distance from the first smart glass, the movement direction of the object, and the movement speed of the object, and controls the first communication unit to transmit the visual information of the object with a relatively high priority to the second smart glass. Thus, the smart glass system can set the priority according to, for example, the movement characteristics of the object.

[0085] (Aspect 6) In a smart glass system according to one aspect, the first control unit of the first smart glass acquires the second position information about the position of its own device acquired by the second smart glass and the movement direction information indicating the movement direction based on the movement trajectory of the position from the second smart glass, determines the priority of the object based on the movement characteristics of the object and the position of the second smart glass recorded in the second position information and the movement direction recorded in the movement direction information, and controls the first communication unit to transmit the visual information of the object with a relatively high priority to the second smart glass. Thus, in the smart glass system, the first smart glass can acquire the position and movement direction of the second smart glass from the second smart glass to which the visual information is to be transmitted (receive feedback from the second smart glass), and set the priority according to the position and movement direction (feedback from the second smart glass) of the second smart glass.

[0086] (Aspect 7) In a smart glass system according to one aspect, when the second control unit of the second smart glass receives the visual information transmitted from the first smart glass, it may transmit the visual information to one or more other second smart glasses existing around the second smart glass. Thus, the smart glass system can retransmit (re-broadcast) the visual information received by the second smart glass to other second smart glasses.

[0087] (Aspect 8) In a smart glass system of one aspect, each second control unit of the second smart glass and another second smart glass may be able to repeat transmitting visual information to another second smart glass different from its own device until the number of transmissions reaches a preset threshold each time visual information is transmitted. Thereby, the smart glass system can retransmit (rebroadcast) the visual information received by the second smart glass to another second smart glass. In this case, when the smart glass system performs retransmission (rebroadcast) of the received visual information, for example, it follows its own re - distribution algorithm (for example, a flooding algorithm that determines not to rebroadcast visual information that has been broadcast once by its own device), but conditions for re - distribution can be added to enhance efficiency. The smart glass system can, for example, include as additional re - distribution conditions the number of re - distributions, the total transmission distance, the use of position information of the smart glass and the detected object respectively, not retransmitting visual information that the second smart glass did not use (did not output a notice), cooperation with the priority set in the first smart glass (not retransmitting if the priority of the visual information in the second smart glass is relatively low), etc. It can use one of the above - mentioned re - distribution conditions (multiple conditions), or a combination of multiple of the above - mentioned re - distribution conditions (multiple conditions).

[0088] (Aspect 9) In a smart glass system of one aspect, each second control unit of the second smart glass and another second smart glass may stop transmitting visual information to another second smart glass different from its own device when the total transmission distance, which is the sum of one or more of the transmission distances at which the visual information was transmitted, exceeds a preset threshold each time visual information is received. As a result, the smart glass system can retransmit (re-broadcast) the visual information received by the second smart glass to other second smart glasses.

[0089] (Aspect 10) In a smart glass system according to one aspect, the second control unit of the second smart glass acquires, from other second smart glasses, second position information about the position acquired by the other second smart glasses and movement direction information indicating the movement direction based on the movement trajectory of the position, and based on the second position information and the movement direction information, and the movement characteristics of the object recorded in the visual information, determines whether the object is an entity that should be noted for the other second smart glasses, and when it is determined that the object is an entity that should be noted for the other second smart glasses, may transmit the visual information to the other second smart glasses. As a result, in the smart glass system, the second smart glass can acquire in advance the position and movement direction of other second smart glasses different from its own device from other second smart glasses, and recognize the movement characteristics of the other second smart glasses. Thereafter, when the second smart glass acquires visual information from the first smart glass, it can compare the movement characteristics of the object recorded in the visual information with the movement characteristics of the other second smart glasses. When the object approaches other second smart glasses, etc., the second smart glass re-distributes (re-broadcasts) the visual information to the other second smart glasses. As a result, the other second smart glasses can perform an output for paying attention to the object based on the visual information. Similarly, other second smart glasses can re-distribute the visual information to still other second smart glasses in the same manner as the second smart glasses described above. That is, a plurality of second smart glasses can repeatedly re-distribute the visual information. On the other hand, other second smart glasses can, for example, further prevent re-distribution of the smart glasses to still other second smart glasses when the still other second smart glasses and the object move away from each other, based on the movement characteristics of the still other second smart glasses and the movement characteristics of the object recorded in the visual information. In addition, when there are a plurality of first smart glasses that have detected the same object, the second smart glasses can acquire visual information from each of the plurality of first smart glasses. Here, the second smart glasses may determine whether the objects recorded in each piece of visual information are the same or not based on the position information recorded in each of the plurality of pieces of visual information. When the objects recorded in each piece of visual information are the same, the second smart glasses can transmit (re - distribute (transfer)) one of the plurality of pieces of visual information to another second smart glass.

[0090] (Aspect 11) In a smart glass system according to one aspect, each control unit of a plurality of smart glasses acquires position information indicating the position of its own device acquired by each device, and movement direction information indicating the movement direction of each device indicated by the trajectory of that position, determines whether a plurality of smart glasses exist in the vicinity, and when it is determined that a plurality of smart glasses exist in the vicinity, groups the plurality of smart glasses, determines a representative of the group, sets the smart glass that becomes the representative of the group as the first smart glass, and the first control unit of the first smart glass may set other smart glasses for which grouping has not been performed as the second smart glasses and transmit the visual information generated by the first smart glass to the second smart glasses. That is, when a plurality of smart glasses exist in the same area or at a relatively short distance (there are a plurality of smart glass users), the smart glass system groups the smart glasses that detect the same object into one group, and the smart glass that becomes the representative of the group can distribute visual information. In this case, the smart glass system can enable each of the plurality of smart glasses to recognize that they are in the same area based on the detected object and the position and movement direction of the smart glasses, etc. in the first - time distribution of visual information. In this case, the smart glass system can, for example, set the smart glass closest to the detected object or the smart glass closest to the intersection as the representative of the group. In addition, in the smart glass system, other smart glasses different from the representative of the group can stop the distribution of visual information for a certain period of time, or relatively reduce the frequency of distributing visual information. As a result, even when there are multiple smart glasses capable of distributing visual information based on the same object, the smart glass system can prevent redundant distribution packets from flooding the mesh network when each of the multiple smart glasses distributes visual information, and can suppress an increase in communication traffic.

[0091] (Aspect 12) A smart glass system according to one aspect includes a plurality of smart glass terminals each including a smart glass and a mobile terminal communicable with the smart glass, and each of the plurality of smart glass terminals includes a camera, a communication unit, and a control unit. Among the plurality of smart glass terminals, a first smart glass terminal includes a camera that images a direction including the field of view of the user of the first smart glass terminal to generate image information, a first control unit as a control unit that performs image recognition on the image information generated by the camera and extracts an object and an event recorded in the image information, and a first communication unit as a communication unit that transmits visual information recording the content of the object and the event extracted by the first control unit to one or a plurality of second smart glass terminals different from the first smart glass terminal among the plurality of smart glass terminals. The second smart glass terminal includes a second communication unit as a communication unit that receives the visual information transmitted from the first communication unit of the first smart glass terminal, and a second control unit as a control unit that generates predetermined content about the object and the event according to the content of the object recorded in the visual information received by the second communication unit and controls to display the predetermined content on the second smart glass terminal. As a result, even when some functions of the smart glass described in the above-described one aspect are offloaded to a mobile terminal such as a smartphone and a tablet, the smart glass system can achieve the same effect as the smart glass system of the above-described one aspect.

[0092] (Aspect 13) A smart glass of one aspect includes a camera that images a direction including the user's field of view to generate image information, a first control unit that performs image recognition on the image information generated by the camera and extracts an object and an event recorded in the image information, and a first communication unit that transmits visual information recording the content of the object and the event extracted by the first control unit to one or more second smart glasses different from the first smart glass serving as the own device. Thereby, the smart glass can achieve the same effect as the smart glass system of the one aspect described above.

[0093] (Aspect 14) A smart glass of one aspect includes a second communication unit that receives visual information transmitted from a first smart glass different from the own device, and a second control unit that generates, by an AI, predetermined content about an object and an event according to the content of the object recorded in the visual information received by the second communication unit and controls to display the predetermined content on the second smart glass serving as the own device. Thereby, the smart glass can achieve the same effect as the smart glass system of the one aspect described above.

[0094] By using the present disclosure described above, it is possible to contribute to the achievement of Goal 9, "Build the infrastructure for industry and innovation," of the Sustainable Development Goals (SDGs).

Description of Reference Numerals

[0095] 1 Smart glass system 100 Smart glass 100a First smart glass 100b Second smart glass 110 Control unit 110a First control unit 110b Second control unit 121 Camera 122 Communication unit 122a First communication unit 122b Second communication unit 123 memory unit 123 display unit

Claims

1. A smart glass system comprising a plurality of smart glasses capable of communicating with each other, each of the plurality of smart glasses including a camera, a communication unit, and a control unit, wherein a first smart glass among the plurality of smart glasses, includes a camera that captures an image in a direction including the field of view of the user of the first smart glass to generate image information, a first control unit as the control unit that performs image recognition on the image information generated by the camera and extracts objects and events recorded in the image information, a first communication unit as the communication unit that transmits visual information recording the contents of the objects and events extracted by the first control unit to one or more second smart glasses different from the first smart glass among the plurality of smart glasses, and is provided with, the first control unit, continuously extracts the objects and events over time, acquires movement characteristics as the contents of the same object, including the movement direction and speed of the same object among the extracted objects and the distance from the first smart glass to the same object, determines the priority when transmitting the visual information of the same object to the second smart glass according to the movement characteristics, controls the first communication unit to include first position information indicating the position of the own device acquired by the first smart glass in the visual information and transmit the visual information to the second smart glass according to the priority, wherein the second smart glass, includes a second communication unit as the communication unit that receives the visual information transmitted from the first communication unit of the first smart glass, a second control unit as the control unit that generates predetermined contents about the objects and events according to the contents of the objects recorded in the visual information received by the second communication unit and controls to display the predetermined contents on the second smart glass, and is provided with, a smart glass system.

2. The first communication unit of the first smart glass broadcasts the visual information to one or more of the second smart glasses existing around the first smart glass. The smart glass system according to Claim 1.

3. The second control unit of the second smart glass, Upon receiving the visual information transmitted by the first communication unit, based on the movement characteristics of the same object recorded in the visual information, the first position information, and the second position information indicating the position of the own device acquired by the second smart glasses, it is determined whether the same object is an object that should be noted. When determining whether the same object is an object that should be noted, as predetermined content regarding the object and the event, content indicating attention to the same object is generated, and control is performed to display the content indicating attention. The smart glasses system according to claim 1.

4. The first control unit of the first smart glasses When extracting a plurality of the objects, using the movement characteristics of each of the plurality of the objects, it is determined that the priority is relatively high according to at least one of the distance from the first smart glasses, the movement direction of the object, and the movement speed of the object. The first communication unit is controlled to transmit the visual information of the object with a relatively high priority to the second smart glasses. The smart glasses system according to claim 1.

5. The first control unit of the first smart glasses The second position information regarding the position of the own device acquired by the second smart glasses and the movement direction information indicating the movement direction based on the movement trajectory of the position are acquired from the second smart glasses. Based on the movement characteristics of the object, the position of the second smart glasses recorded in the second position information, and the movement direction recorded in the movement direction information, the priority of the object is determined. The first communication unit is controlled to transmit the visual information of the object with a relatively high priority to the second smart glasses. The smart glasses system according to claim 3.

6. The second control unit of the second smart glasses When receiving the visual information transmitted from the first smart glasses, the visual information is transmitted to one or a plurality of other second smart glasses existing around the second smart glasses. The smart glasses system according to claim 3.

7. The second control units of the second smart glasses and the other second smart glasses Each time the visual information is transmitted, the number of transmissions is recorded in the visual information. Until the number of transmissions reaches a preset threshold value, it is possible to repeat the transmission of the visual information to other second smart glasses different from the own device. The smart glasses system according to claim 6.

8. Each of the second smart glasses and the other second smart glasses has a second control unit, when receiving visual information, records the transmission distance at which the visual information was transmitted in the visual information, when the total transmission distance, which is the sum of one or more of the transmission distances, exceeds a preset threshold, stops transmitting visual information to other second smart glasses different from the own device The smart glass system according to claim 6.

9. The second control unit of the second smart glass acquires, from the other second smart glasses, second position information about a position acquired by the other second smart glasses and movement direction information indicating a movement direction based on a movement trajectory of the position, determines whether the object is an entity that should be noted for the other second smart glasses based on the second position information and the movement direction information and the movement characteristics of the object recorded in the visual information, when it is determined that the object is an entity that should be noted for the other second smart glasses, transmits visual information to the other second smart glasses The smart glass system according to claim 6.

10. A smart glass system including a plurality of smart glasses capable of communicating with each other, each of the plurality of smart glasses including a camera, a communication unit, and a control unit, a first smart glass among the plurality of smart glasses includes a camera that captures an image in a direction including the field of view of the user of the first smart glass and generates image information, a first control unit as the control unit that performs image recognition on the image information generated by the camera and extracts an object and an event recorded in the image information, a first communication unit as the communication unit that transmits visual information recording the content of the object and the event extracted by the first control unit to one or more second smart glasses different from the first smart glass among the plurality of smart glasses, and the second smart glass includes a second communication unit as the communication unit that receives the visual information transmitted from the first communication unit of the first smart glass, a second control unit as the control unit that generates predetermined content about the object and the event according to the content of the object recorded in the visual information received by the second communication unit and controls to display the predetermined content on the second smart glass, and each of the control units of the plurality of smart glasses Obtain position information indicating the position of the own device acquired by each device, and movement direction information indicating the movement direction of each device indicated by the trajectory of the position, Determine whether a plurality of the smart glasses exist in the surroundings, When it is determined that a plurality of the smart glasses exist in the surroundings, group the plurality of the smart glasses and determine a representative of the group, Set the smart glass serving as the representative of the group as the first smart glass, The first control unit of the first smart glass designates other smart glasses for which the grouping has not been performed as the second smart glasses, Transmit the visual information generated by the first smart glass to the second smart glass, Smart glass system.

11. A smart glass system including a plurality of smart glass terminals each including a smart glass and a mobile terminal capable of communicating with the smart glass, wherein each of the plurality of smart glass terminals includes a camera, a communication unit, and a control unit, Among the plurality of smart glass terminals, a first smart glass terminal, A camera that images a direction including the field of view of the user of the first smart glass terminal to generate image information, A first control unit as the control unit that performs image recognition on the image information generated by the camera and extracts an object and an event recorded in the image information, A first communication unit as the communication unit that transmits visual information recording the content of the object and the event extracted by the first control unit to one or a plurality of second smart glass terminals different from the first smart glass terminal among the plurality of smart glass terminals, Comprises, The first control unit, Extract the object and the event continuously in time, Obtain movement characteristics as the content of the same object including the movement direction and movement speed of the same object among the extracted objects and the distance from the first smart glass terminal to the same object, Determine the priority when transmitting the visual information of the same object to the second smart glass terminal according to the movement characteristics, Include first position information indicating the position of the own device acquired by the first smart glass terminal in the visual information, and control the first communication unit to transmit the visual information to the second smart glass terminal according to the priority, The second smart glass terminal, A second communication unit as the communication unit that receives the visual information transmitted from the first communication unit of the first smart glass terminal, A second control unit as the control unit that generates predetermined content about the object and event according to the content of the object recorded in the visual information received by the second communication unit, and controls to display the predetermined content on the second smart glass terminal. Comprising A smart glass system.

12. A camera that captures an image in a direction including the user's field of view to generate image information, A first control unit that performs image recognition on the image information generated by the camera and extracts an object and an event recorded in the image information, A first communication unit that transmits visual information recording the content of the object and event extracted by the first control unit to one or more second smart glasses different from the first smart glass serving as the own device. The first control unit Extracts the object and event continuously in time, Obtains movement characteristics as the content of the same object including the movement direction and movement speed of the same object among the extracted objects and the distance from the first smart glass to the same object, Determines the priority when transmitting the visual information of the same object to the second smart glass according to the movement characteristics, Controls the first communication unit to include first position information indicating the position of the own device acquired by the first smart glass in the visual information and transmit the visual information to the second smart glass according to the priority. A smart glass.

13. A second communication unit that receives the visual information transmitted from the first smart glass, A second control unit that generates, by AI, content indicating predetermined content about the object and event according to the content of the object recorded in the visual information received by the second communication unit, and controls to display the predetermined content on the second smart glass serving as the own device. The smart glass according to claim 12, comprising.

Images