Information processing device, information processing method and information processing program

By calculating the characteristic scores of ecosystem components and controlling the display of visualization information, the problem of inaccurate association between ecosystem components and visualization information in AR technology is solved, thereby improving the usability of AR technology and the accuracy of user operation.

CN115210769BActive Publication Date: 2026-11-10SONY GROUP CORP
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202180017788.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2020-03-06
Filing Date
2021-01-26
Publication Date
2026-11-10
Estimated Expiration
2041-01-26

AI Technical Summary

Technical Problem

In conventional AR technology, users find it difficult to accurately grasp the relationship between ecosystem components and visualized information, especially when ecosystem components are densely arranged, resulting in insufficient usability.

Method used

An information processing device is used to calculate the characteristic scores of ecosystem components, and the display of visual information is controlled based on the scores. The calculation unit and the presentation control unit realize the precise correspondence between ecosystem components and visual information, and AR technology is used to display virtual objects of ecosystem components on the terminal device.

Benefits of technology

It improves the accuracy of users' understanding of the relationship between ecosystem components and visualization information, enhances the usability of AR technology, and enables users to more accurately understand and manipulate the complexity and location of ecosystem components.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115210769B_ABST
    Figure CN115210769B_ABST
Patent Text Reader

Abstract

This invention promotes further improvements in usability. An information processing apparatus (10) according to an embodiment includes: a calculation unit (1122) that calculates a score indicating the characteristics of a target object; and a presentation control unit (1123) that performs processing for displaying visual information for the target object based on height information corresponding to the score calculated by the calculation unit (1122).
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This disclosure relates to information processing apparatus, information processing methods, and information processing procedures. Background Technology

[0002] As an example of agricultural methods, there is one called synecoculture (registered trademark), which is based on no tillage, no fertilization, and no pesticide use. Synecoculture (registered trademark) is influenced by various ecosystem components, making it difficult for workers to learn in a short period of time, thus requiring the assistance of technicians. Therefore, in recent years, technologies for remotely assisting agricultural workers with field technicians (e.g., fields or farms) have gained attention.

[0003] With the assistance of the Co-production Farming Method (registered trademark), there is a need to provide accurate guidance to workers in remote locations such as farmland. Therefore, in recent years, augmented reality (AR) has attracted attention as a technology for providing accurate vision-based guidance to workers in remote locations.

[0004] Reference List

[0005] Patent documents

[0006] Patent Document 1: WO 2017 / 061281 A Summary of the Invention

[0007] Technical issues

[0008] However, in conventional AR technologies, users may find it difficult to accurately grasp the relationship between ecosystem components and visualized information. Therefore, there is room for improvement in usability.

[0009] In view of this, the present disclosure proposes novel and improved information processing apparatus, information processing methods and information processing procedures that can promote further improvement in usability.

[0010] Solution to the problem

[0011] According to this disclosure, an information processing apparatus is provided, comprising: a calculation unit that calculates a score indicating the characteristics of a target object; and a presentation control unit that performs processing for displaying visual information for the target object based on height information corresponding to the score calculated by the calculation unit. Attached Figure Description

[0012] Figure 1 This is a diagram illustrating a configuration example of an information processing system according to an implementation method.

[0013] Figure 2This is a diagram illustrating an implementation example of an information processing system according to an embodiment.

[0014] Figure 3 This is a diagram illustrating an implementation example of an information processing system according to an embodiment.

[0015] Figure 4 This is a diagram illustrating an implementation example of an information processing system according to an embodiment.

[0016] Figure 5 This is a diagram illustrating an overview of the functions of an information processing system according to an embodiment.

[0017] Figure 6 This is a block diagram illustrating a configuration example of an information processing system according to an implementation method.

[0018] Figure 7 This is a diagram illustrating an overview of the functions of an information processing system according to an embodiment.

[0019] Figure 8 This is a diagram illustrating an example of a variation of the visualization information according to an implementation method.

[0020] Figure 9 This is a diagram illustrating an example of a storage unit according to an embodiment.

[0021] Figure 10 This is a flowchart illustrating the processing flow in an information processing apparatus according to an embodiment.

[0022] Figure 11 This is a flowchart illustrating the processing flow in an information processing apparatus according to an embodiment.

[0023] Figure 12 This is a flowchart illustrating the processing flow in an information processing apparatus according to an embodiment.

[0024] Figure 13 This is a flowchart illustrating the processing flow in an information processing apparatus according to an embodiment.

[0025] Figure 14 This is a diagram illustrating an example of a variation of the visualization information according to an implementation method.

[0026] Figure 15 This is a hardware configuration diagram illustrating an example of a computer that performs the functions of an information processing device. Detailed Implementation

[0027] Preferred embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. Note that redundant descriptions will be omitted from this specification and the drawings by assigning the same reference numerals to components having substantially the same functional configuration.

[0028] Note that the descriptions will be provided in the following order.

[0029] 1. One implementation method of this disclosure

[0030] 1.1. Introduction

[0031] 1.2. Configuration of Information Processing System

[0032] 2. Implementation Example of an Information Processing System

[0033] 2.1. Identification of farmland and relocation to the workplace

[0034] 2.2. Work

[0035] 2.3. Confirmation after work

[0036] 3. Functions of the information processing system

[0037] 3.1. Functional Overview

[0038] 3.2. Function Configuration Example

[0039] 3.3. Information Processing System Processing

[0040] 3.4. Variations in processing

[0041] 4. Hardware Configuration Example

[0042] 5. Summary

[0043] <<1. One Implementation of the Contents of this Disclosure>>

[0044] <1.1. Introduction>

[0045] Regarding Co-cultivation Farming (registered trademark), understanding the compatibility between ecosystem components such as plants and the complexity of the entire ecosystem is also important. To this end, techniques such as using AR (Augmented Reality) to apply visual information, such as markers, to ecosystem components have attracted attention. However, in Co-cultivation Farming (registered trademark), there are situations where a wide variety of ecosystem components are densely arranged, and in such cases, visual information is given to a large number of ecosystem components. This can make it difficult for users to accurately grasp the relationship between ecosystem components and visual information.

[0046] Furthermore, while techniques using color information (e.g., heat maps) to display visualizations indicating conceptual information (e.g., altitude or temperature) associated with each ecosystem construct have garnered attention, the applied color information can sometimes obscure or hide ecosystem constructs. Additionally, although techniques for displaying ecosystem constructs without obscuring or hiding them—e.g., displaying them in a different space than the space where they are currently displayed—have also attracted attention, in some cases the display and presentation positions of ecosystem constructs and visualizations can be misaligned. This can lead to an inability to grasp the spatial relative positions of ecosystem constructs and visualizations, making it difficult for users to accurately understand the association between them.

[0047] On the other hand, as mentioned above, when using conventional AR technology, users may find it difficult to accurately grasp the relationship between ecosystem components and visualized information. Therefore, there is room for improvement in usability.

[0048] In view of this, the present disclosure proposes novel and improved information processing apparatus, information processing methods and information processing procedures that can promote further improvement in usability.

[0049] In the following description, farm workers are appropriately referred to as "users." A user can be someone experiencing the AR experience as a farm worker. In this implementation, the user is not an actual farm worker, but rather someone experiencing the AR experience as a farm worker. Furthermore, in the following text, the farm technician guiding the user will be appropriately referred to as an "instructor." An instructor can be a guide or mentor who guides a user experiencing the AR experience as a farm worker. In this implementation, the instructor is a guide or mentor who guides a user experiencing the AR experience as a farm worker, and not a technician in an actual farm.

[0050] In the following text, the target object according to the implementation method may be an object (virtual object) provided in AR representation at a location corresponding to real space or an actual object (real object).

[0051] In the following text, a predetermined area comprising multiple target objects is appropriately set as the target range. Note that when the target object is a virtual object, the target range is the display range described below.

[0052] <1.2. Configuration of Information Processing System>

[0053] The configuration of the information processing system 1 according to the implementation method will be described. Figure 1 This is a diagram illustrating a configuration example of information processing system 1. (See diagram for example.) Figure 1As shown, the information processing system 1 includes an information processing device 10, a terminal device 20, and an information providing device 30. The information processing device 10 can be connected to various types of devices. For example, the terminal device 20 and the information providing device 30 are connected to the information processing device 10 and perform information exchange between the devices. The information processing device 10 is wirelessly connected to the terminal device 20 and the information providing device 30. For example, the information processing device 10 uses Bluetooth (registered trademark) to perform short-range wireless communication with the terminal device 20 and the information providing device 30. Note that the terminal device 20 and the information providing device 30 can be connected to the information processing device 10 via a wired channel or via a network.

[0054] (1) Information processing device 10

[0055] The information processing device 10 is an information processing device that performs processing for displaying visual information about a target object. Specifically, the information processing device 10 calculates a score (complexity) indicating the characteristics of the target object. Based on the calculated score, the information processing device 10 provides control information for displaying visual information about the target object.

[0056] Furthermore, the information processing device 10 also has the function of controlling the overall operation of the information processing system 1. For example, the information processing device 10 controls the overall operation of the information processing system 1 based on information exchanged between individual devices. Specifically, for example, the information processing device 10 controls the visual information displayed on the terminal device 20 based on information received from the information providing device 30.

[0057] The information processing device 10 is implemented by a personal computer (PC), a workstation (WS), or the like. Note that the information processing device 10 is not limited to PCs, WSs, etc. For example, the information processing device 10 can be an information processing device such as a PC or a WS equipped with functions as the information processing device 10.

[0058] (2) Terminal device 20

[0059] Terminal device 20 is a wearable device such as HoloLens, which can output AR representations. Alternatively, terminal device 20 can be a terminal device such as a smartphone capable of outputting mobile AR representations including ARCore (registered trademark) or ARKit (registered trademark). Furthermore, terminal device 20 can be a video-based AR device or XR device such as Varjo's XR-1 (registered trademark).

[0060] For example, terminal device 20 displays information about visual information based on control information provided from information processing device 10.

[0061] (3) Information providing device 30

[0062] The information providing device 30 is an information processing device that provides information about the target range to the information processing device 10. For example, the information processing device 10 provides information about the target range based on information obtained about information related to the target range.

[0063] The information providing device 30 is implemented by a PC, WS, etc. Note that the information providing device 30 is not limited to PCs, WS, etc. For example, the information providing device 30 can be an information processing device such as a PC or WS equipped with the functions of an information providing device 30 as an application.

[0064] <<Examples of Information Processing System Implementation>>

[0065] The configuration of information processing system 1 has been described above. Next, an implementation example of information processing system 1 will be described. In this implementation, the farmland is not actual farmland, but a simulated farmland used for the AR experience, and is therefore appropriately referred to as "AR farmland." Furthermore, in this implementation, it is assumed that user U11 wears see-through glasses that limit the field of view. Additionally, in this implementation, it is assumed that a workplace exists within the farmland. Furthermore, virtual objects will be described as examples of target objects.

[0066] <2.1. Identification of farmland and its relocation to the workplace>

[0067] Figure 2 The diagram shows scene FS1 where user U11, as the target of the AR experience, confirms the farmland, and scene FS2 where user U11 moves to the workplace. Figure 2 This includes an action scene US11 that instructs the user U11's actions during the AR experience, and an AR scene AS11 that instructs the AR representation displayed along with the user U11's actions. The action scene US and the AR scene AS will be described in relation to each other below. Note that action images UR11 to UR14 correspond to AR images AR11 to AR14, respectively.

[0068] Figure 2 First, a guidance scene GS11 is shown, in which user U11 is instructed, “Let’s get started. Hold the tomato seedling.” Action image UR11 is an image showing user U11 holding a tomato seedling and waiting in a location slightly away from the farmland. AR image AR11 is an image displayed on terminal device 20 showing an AR representation. AR image AR11 is in a state without AR representation and therefore displays the entire background of the real space. Next, in response to the instructor’s action, information processing system 1 displays virtual objects of vegetation in the AR farmland in the AR representation (S11). Subsequently, user U11 proceeds to guidance scene GS12, where he is instructed, “This is a view of the entire farmland.”

[0069] Action image UR12 is an image instructing user U11 to grasp the entire farmland by looking down at it. AR image AR12 is an image indicating the AR representation displayed on terminal device 20. In AR image AR12, virtual objects of vegetation are displayed in the AR farmland. For example, AR image AR12 includes the display of virtual objects such as tomato OB11 and carrot OB12. In AR image AR12, for ease of description, virtual objects OB11 and OB12, which are examples of virtual objects, are labeled with reference numerals, but other virtual objects included in AR image AR12 may also be assigned reference numerals. Note that the scene obtained by combining guidance scene GS11 with guidance scene GS12 is scene FS1. Subsequently, user U11 moves to guidance scene GS13, which is instructed to "approach today's workplace".

[0070] Action image UR13 is an image indicating a scene where user U11 approaches the workplace. Information processing system 1 performs a process (S12) to restrict the display range of virtual objects representing vegetation based on user U11's actions. AR image AR13 is an image representing AR displayed on terminal device 20. AR image AR13 has a restricted display range for virtual objects representing vegetation displayed in AR image AR12. For example, in AR image AR13, the display range of virtual objects is restricted so that only information that allows user U11 to process within a predetermined time (e.g., within the time corresponding to daily working hours) is displayed. Due to this restriction, information processing system 1 can accurately guide user U11 to the workplace. AR image AR13 includes the display of, for example, virtual objects OB13 (potatoes) and OB14 (cabbages). In AR image AR13, for ease of description, virtual objects OB13 and OB14, as examples of virtual objects, are labeled with reference numerals; however, reference numerals may also be assigned to other virtual objects included in AR image AR13. Next, in response to the instructor's action, the information processing system 1 executes an AR representation of a virtual object as visualization information that visualizes the complexity (diversity) of the vegetation (S13). Note that this complexity corresponds to the score of the characteristic indicating the target object or the height information corresponding to the score of the characteristic indicating the target object. Subsequently, the user U11 proceeds to the guidance scene GS14, which includes the instruction "This indicates the complexity of the vegetation. Let's plant seedlings in areas with low complexity."

[0071] Action image UR14 is an image of a scene that instructs user U11 to confirm the location for improvement. AR image AR14 is an image that instructs user U11 to display an AR representation on terminal device 20. AR image AR14 displays a virtual object OB15 in AR representation to visualize the complexity of vegetation. AR image AR14, for example, displays the virtual object OB15 as a grid 3D graphic indicating the complexity of vegetation in AR representation. The virtual object OB15 indicates the complexity of vegetation based on the height of the grid 3D graphic. For example, the virtual object OB15 indicates how the virtual object relates to the vegetation. Furthermore, the downward indentation of the virtual object OB15 indicates locations with less vegetation, i.e., places where user U11 needs to work. This allows information processing system 1 to accurately instruct user U11 on the locations where work needs to be done. Note that scene FS2 is a combination of guidance scene GS13 and guidance scene GS14. Processing proceeds to Figure 3 The scene shown.

[0072] <2.2. Work>

[0073] Figure 3 This is a diagram of scene FS3 showing the AR experience performed by user U11. Figure 3 This includes: an action scene US12 that indicates the action of user U11; and an AR scene AS12 that indicates the AR representation displayed in association with the action of user U11. In the following description, the action scene US and the AR scene AS will be described in relation to each other. Note that action images UR15 to UR18 correspond to AR images AR15 to AR18, respectively.

[0074] Figure 3The system includes a guidance scenario GS15, in which user U11 is instructed to “Let’s crouch down and work. Be careful not to damage the roots.” Action image UR15 is an image showing user U11 crouching and waiting. AR image AR15 is an image indicating an AR representation displayed on terminal device 20. In response to the instructor’s actions, AR image AR15 displays virtual objects that visualize the extent of the vegetation’s roots. For example, AR image AR15 displays virtual objects OB16 to OB18 that visualize the extent of the vegetation’s roots. This allows information processing system 1 to accurately indicate to user U11 the vegetation roots that should not be damaged. In AR image AR15, for ease of description, virtual objects OB16 to OB18, as examples of virtual objects, are labeled with reference numerals; however, reference numerals can also be assigned to other virtual objects included in AR image AR15. Next, in response to the instructor’s actions, information processing system 1 executes a real-time AR representation of the virtual objects that visualizes the instructor’s hand movements (S14). This allows the instructor to provide accurate directional guidance even from a remote location. User U11 then proceeds to scenario GS16, which is guided to “This seems like a good place to plant seedlings.”

[0075] Action image UR16 is an image showing a scene where user U11 plants tomato seedlings at a location indicated by an instructor. AR image AR16 is an image indicating an AR representation displayed on terminal device 20. In response to the instructor's operation, AR image AR16 performs real-time display of virtual objects that visualize the movement of the instructor's hand. For example, AR image AR16 displays virtual object OB19 that visualizes the movement of the instructor's hand. Virtual object OB19 changes in real time according to the movement of the instructor's hand. Furthermore, AR image AR16 displays virtual objects that visualize the area where work needs to be done according to the instructor's operation. For example, AR image AR16 displays virtual object OB20 that visualizes the area where work needs to be done. With this configuration, information processing system 1 can accurately provide directional guidance to the area where work needs to be done by performing AR representation of the part that needs to be done in conjunction with the movement of the instructor's hand. In AR image AR16, for ease of description, virtual objects OB19 and OB20, which are examples of virtual objects, are labeled with reference numerals, but reference numerals may also be assigned to other virtual objects included in AR image AR16. Next, in response to the instructor's actions, the information processing system 1 executes a real-time AR representation of the virtual object that visualizes the detailed or model behavior indicated by the instructor's hand movements (S15). With this configuration, the instructor can accurately guide working methods, including subtle nuances. Subsequently, the user U11 proceeds to the guidance scenario GS17, which is instructed to "cover with soil like this."

[0076] Action image UR17 is an image showing a scene where user U11 covers seedlings with soil according to the instructor's instructions. AR image AR17 is an image indicating the AR representation displayed on terminal device 20. In response to the instructor's operation, AR image AR17 performs real-time display of virtual objects that visualize the movement of the instructor's hands. For example, AR image AR17 displays virtual objects OB19 and OB21 that visualize the movement of the instructor's hands. Virtual object OB19 changes in real-time according to the movement of the instructor's right hand. Virtual object OB21 changes in real-time according to the movement of the instructor's left hand. With this configuration, information processing system 1 performs AR representation of the movement of the instructor's hands, making pointing guidance more accurate. In AR image AR17, for ease of description, virtual objects OB19 and OB21, as examples of virtual objects, are labeled with reference numerals, but reference numerals can also be assigned to other virtual objects included in AR image AR17. Next, in response to the instructor's operation, information processing system 1 performs real-time AR representation of virtual objects that visualize the feedback indicated by the movement of the instructor's hands (S16). Using this operation, the instructor can reassure user U11 through feedback. User U11 then proceeds to the instruction scenario GS18, where the instruction "looks good."

[0077] Action image UR18 is an image of a scene instructing user U11 to acknowledge feedback from the instructor and stand up. AR image AR18 is an image indicating an AR representation displayed on terminal device 20. In response to the instructor's action, AR image AR18 performs a real-time display of a virtual object that visualizes the movement of the instructor's hand providing feedback. For example, AR image AR18 displays a virtual object OB19 that visualizes the movement of the instructor's hand providing feedback. Virtual object OB19 changes in real-time according to the movement of the instructor's hand providing feedback. With this configuration, information processing system 1 performs an AR representation of the instructor's feedback, enabling more accurate pointing guidance. In AR image AR18, for ease of description, virtual object OB19, as an example of a virtual object, is labeled with reference numerals, but reference numerals can also be assigned to other virtual objects included in AR image AR18. Note that scene FS3 is a combination of guidance scenes GS15 to GS18. Processing proceeds to... Figure 4 The scene shown.

[0078] <Confirmation after work>

[0079] Figure 4 This is a diagram of scenario FS4, which shows the confirmation of the work performed by user U11. Figure 4This includes: an action scene US13 indicating the actions of user U11; and an AR scene AS13 indicating the AR representation displayed in association with the actions of user U11. The action scene US and the AR scene AS will be described in relation to each other below. Note that action images UR19 to UR22 correspond to AR images AR19 to AR22, respectively.

[0080] Figure 4 The diagram illustrates a guidance scenario GS19 in which user U11 is instructed to have "achieved greater diversity." Action image UR19 is an image showing user U11 browsing through sections of the scene where work has been performed. AR image AR19 is an image indicating an AR representation displayed on terminal device 20. In response to the instructor's actions, AR image AR19 displays a virtual object OB22 in AR representation that visualizes the complexity of vegetation. AR image AR19 displays, for example, the virtual object OB22 as a grid-like 3D graphic indicating the complexity of vegetation. Virtual object OB22 indicates complexity based on height differences. For example, virtual object OB22 indicates that location PT11 has high complexity and abundant vegetation. Furthermore, virtual object OB22 indicates that location PT12 has low complexity and infrequent (barren) vegetation. Additionally, location PT13 is where user U11 planted seedlings in scene FS3. Virtual object OB22 indicates that location PT13 now has higher complexity and more abundant vegetation. Through this configuration, information processing system 1 allows user U11 to perceive the effects of their work. In AR image AR19, for ease of description, virtual object OB22, as an example of a virtual object, is labeled with an abbreviation. However, other virtual objects included in AR image AR19 can also be assigned abbreviations. Next, in response to the instructor's operation, information processing system 1 displays virtual objects in AR representation that visualize the complexity of the entire farmland (S17). Through this operation, the instructor can make it easier for user U11 to find other areas that need improvement. Subsequently, user U11 moves to the guidance scene GS20, which is guided to "the entire farmland looks good".

[0081] Action image UR20 is an image showing a scene where user U11 looks out from a location slightly away from the farmland. AR image AR20 is an image representing an AR representation displayed on terminal device 20. In response to the instructor's operation, AR image AR20 displays a virtual object OB23 in AR representation that visualizes the complexity of vegetation throughout the farmland. For example, AR image AR20 displays virtual object OB23 in AR representation as a grid-like 3D graphic indicating the complexity of vegetation throughout the farmland. This allows information processing system 1 to accurately indicate to user U11 other locations that need improvement. In AR image AR20, for ease of description, virtual object OB23, as an example of a virtual object, is labeled with reference numerals, but reference numerals can also be assigned to other virtual objects included in AR image AR20. Next, in response to the instructor's operation, information processing system 1 displays a virtual object in AR representation that visualizes the predicted future degree of vegetation growth (S18). With this configuration, the instructor can increase the motivation of user U11. User U11 then proceeds to the instruction scene GS21, which reads, "Pay attention to growth. Let's see how it grows two months from now."

[0082] Action image UR21 is an image instructing user U11 to observe the scene of the entire farmland. AR image AR21 is an image instructing user U11 to observe the AR representation displayed on terminal device 20. In response to the instructor's operation, AR image AR21 displays virtual objects that visualize the predicted future growth of vegetation. For example, AR image AR21 displays virtual objects OB24 to OB26, etc., that visualize the predicted future growth of vegetation. With this configuration, information processing system 1 can further enhance the motivation of user U11. In AR image AR21, for ease of description, virtual objects OB24 to OB26, which are examples of virtual objects, are labeled with reference numerals, but reference numerals can also be assigned to other virtual objects included in AR image AR21. Next, in response to the instructor's operation, information processing system 1 displays virtual objects in AR representation that visualize the predicted future growth of vegetation at the predicted harvest time (S19). With this configuration, the instructor can make it easier for user U11 to determine the harvest time. Subsequently, user U11 proceeds to the instruction scene GS22, which instructs "It will grow like this at harvest. It can be harvested at this size."

[0083] Action image UR22 is an image instructing user U11 to observe the scene of the entire farmland. AR image AR22 is an image instructing user U11 to observe the scene in AR representation displayed on terminal device 20. In response to the instructor's operation, AR image AR22 displays virtual objects such as OB27 in AR representation to visualize the predicted growth of vegetation at harvest time. With this configuration, information processing system 1 can further enhance user U11's enthusiasm for harvesting. In AR image AR22, for ease of description, virtual object OB27, as an example of a virtual object, is labeled with reference numerals, but reference numerals can also be assigned to other virtual objects included in AR image AR22. Note that scene FS4 is a combination of instruction scenes GS19 to GS22.

[0084] The implementation examples of information processing system 1 have been described above with reference to scenarios FS1 to FS4.

[0085] <<Functions of Information Processing Systems>>

[0086] Next, the functions of information processing system 1 will be described.

[0087] <3.1. Functional Overview>

[0088] Figure 5 This is a diagram illustrating the general functionality of the information processing system 1 according to an embodiment. Figure 5 This illustration shows a scenario where user U11 sets their gaze to a target area TG11, including the target object TB11. Furthermore, visualization information KJ11 is displayed for the target area TG11. Here, distance SD11 indicates the relative horizontal distance between user U11 and the target area TG11. Height SH11 indicates the height of user U11's viewpoint relative to the ground. Height SH12 indicates the reference height of the visualization information KJ11 to be displayed relative to the ground. Height SH13 indicates the height of the visualization information KJ11 at location PT11 relative to the reference height of the visualization information KJ11. Note that height SH13 indicates the irregularity (complexity) of the visualization information KJ11. Angle SK11 indicates the angle of view that user U11 feels comfortable moving their eyes while pointing forward. For example, angle SK11 is a visual field angle of 30 degrees or less. Angle SK12 indicates the angle of view that user U11 can look upwards with eye movement while pointing forward. For example, angle SK12 is a visual field angle of 5 degrees or less. Line of sight SS11 indicates the line of sight of user U11 in the direction horizontal to the ground from user U11's viewpoint. Line of sight SS12 indicates the line of sight of user U11 in the direction of angle SK11 from user U11's viewpoint. Line of sight SS13 indicates the line of sight of user U11 in the direction of angle SK12 from user U11's viewpoint.

[0089] <3.2. Function Configuration Example>

[0090] Figure 6 This is a block diagram illustrating an example of the functional configuration of the information processing system 1 according to an embodiment.

[0091] (1) Information processing device 10

[0092] like Figure 6 As shown, the information processing device 10 includes a communication unit 100, a control unit 110, and a storage unit 120. Note that the information processing device 10 includes at least the control unit 110.

[0093] (1-1) Communication Unit 100

[0094] The communication unit 100 has the function of communicating with an external device. For example, when communicating with an external device, the communication unit 100 outputs information received from the external device to the control unit 110. Specifically, the communication unit 100 outputs information received from the information providing device 30 to the control unit 110. For example, the communication unit 100 outputs information about the target range to the control unit 110.

[0095] For example, when communicating with an external device, the communication unit 100 transmits information input from the control unit 110 to the external device. Specifically, the communication unit 100 transmits information about the acquisition of target range information input from the control unit 110 to the information providing device 30.

[0096] (1-2) Control Unit 110

[0097] The control unit 110 has the function of controlling the operation of the information processing device 10. For example, the control unit 110 performs information processing for displaying visual information about the target object based on a score indicating the characteristics of the target object.

[0098] To achieve the above functions, the control unit 110 includes an acquisition unit 111, a processing unit 112, and an output unit 113, such as... Figure 6 As shown.

[0099] • Acquisition Unit 111

[0100] The acquisition unit 111 has the function of acquiring information for displaying visual information about a target object. For example, the acquisition unit 111 acquires sensor information transmitted from the terminal device 20 via the communication unit 100. For example, the acquisition unit 111 acquires sensor information, such as acceleration information, gyroscope information, Global Positioning System (GPS) information, and geomagnetic information, for specifying the location of the terminal device 20.

[0101] For example, the acquisition unit 111 acquires information about the target range transmitted from the information providing device 30 via the communication unit 100. For example, the acquisition unit 111 acquires information about multiple processing objects included in the target range.

[0102] Processing unit 112

[0103] The processing unit 112 has the function of controlling the processing of the information processing device 10. For example... Figure 6 As shown, the processing unit 112 includes a position control unit 1121, a calculation unit 1122, a presentation control unit 1123, and a presentation creation unit 1124.

[0104] • Position control unit 1121

[0105] The location control unit 1121 has the function of determining information about the location of a user having the terminal device 20. The location control unit 1121 determines the user's location information based on the information about the location of the terminal device 20. The location control unit 1121 determines, for example, information about the user's relative position to a target object. For example, the location control unit 1121 determines the relative horizontal distance between the target object and the user. Specifically, the location control unit 1121 determines the user's location information based on at least one of acceleration information, gyroscope information, GPS information, and geomagnetic information. Furthermore, the location control unit 1121 determines the user's location information, for example, based on a target area. Specifically, the location control unit 1121 determines the user's location information based on the location where a straight line connecting the center of the target area and the user intersects one side of the target area.

[0106] • Calculation Unit 1122

[0107] The calculation unit 1122 has the function of calculating scores indicating the characteristics of target objects. The calculation unit 1122 calculates the score for each of multiple target objects included in the target range. Furthermore, the calculation unit 1122 can calculate scores indicating the relationships between multiple target objects included in the target range. For example, the calculation unit 1122 can calculate scores indicating the relationships between target objects that are adjacent to each other. Specifically, the calculation unit 1122 can calculate scores indicating the relationships between multiple target objects, such as interpersonal relationships in a town, product management in a warehouse, personnel allocation in a workplace such as an office, or the level of activity in an event venue. In this case, the target object is an object such as a person or a product in a specific environment. Note that when the target object is related to vegetation, the calculation unit 1122 can calculate scores indicating the characteristics of the target object related to vegetation.

[0108] For example, the calculation unit 1122 calculates a score based on information about the target range transmitted from the information providing device 30 via the communication unit 100.

[0109] The calculation unit 1122 converts the score into height information corresponding to the calculated score. For example, the calculation unit 1122 converts the calculated score into height information based on association information that pre-associates the score with the height information.

[0110] • Presentation control unit 1123

[0111] The presentation control unit 1123 has the function of determining information about the height of the visual information displayed for the target object. For example, the presentation control unit 1123 determines information about the height at which the user can comfortably view the visual information. For example, the presentation control unit 1123 determines the information about the height at which the user can comfortably view the visual information based on information about the user's position relative to the target object, information about the height of the user's viewpoint, and information about the user's field of view. Specifically, the presentation control unit 1123 determines the height of the visual information to be displayed to fall within the viewing angle of the display angle of the terminal device 20, which is determined based on information about the user's position relative to the target object, information about the height of the user's viewpoint, and information about the user's field of view.

[0112] The presentation control unit 1123 determines information about the height of the visual information based on a target field of view that indicates the user's field of view to comfortably view the visual information. Formula (1) below is an example of a calculation formula used to determine the height of the visual information so that the user's field of view falls within the target field of view. Note that in formula (1) below, the upper limit of the target field of view is 5 degrees above the user's viewpoint, and the lower limit is 30 degrees below the user's viewpoint.

[0113]

[0114] In the formula, Hobj represents the height of the visualized information. Heye represents the height of the user's viewpoint. Ddis represents the distance between the user and the target area.

[0115] The presentation control unit 1123 determines information regarding the reference height of the visualization information. Note that, for example, the reference height of the visualization information is a reference value (e.g., average) of the height of the visualization information at each location. For example, the presentation control unit 1123 sets the reference height of the visualization information based on the height of the target object included in the target range. Specifically, the presentation control unit 1123 sets the reference height of the visualization information based on the height of the target object included in the target range, specifically, the height based on the maximum, average, mode, or minimum value of the target object's height.

[0116] Figure 7 An example of the reference height for visualization information determined by the presentation control unit 1123 is shown. Height SH21 is the height of target object TB12, which has the smallest height among the target objects included in the target range TG21. The presentation creation unit 1124 performs processing for displaying visualization information KJ21 based on height SH21. Height SH22 is the mode of the heights of the target objects included in the target range TG21. Specifically, height SH22 is the height of target objects TB11, TB13, and TB14. The presentation creation unit 1124 performs processing for displaying visualization information KJ22 based on height SH22. Height SH23 is the average (mean) of the heights of the target objects included in the target range TG21. Specifically, height SH23 is the average height of target objects TB11 to TB16. The presentation creation unit 1124 performs processing for displaying visualization information KJ23 based on height SH23. Height SH24 is the height of target object TB15, which has the largest height among the target objects included in the target range TG24. The presentation creation unit 1124 performs processing for displaying visual information KJ24 based on the height SH24.

[0117] The presentation control unit 1123 determines an offset for determining the reference height of the visualization information. The offset is used to adjust the height to a level where the user can comfortably view the target object and the visualization information. For example, the offset is used to display the visualization information at a predetermined height distance away from the target object in the upward direction. Specifically, the offset is the height used to display the visualization information at a predetermined distance, which is a distance at which the user can comfortably view the target object. The offset is determined based on the height of the target object and the height of the visualization information to be displayed. The presentation control unit 1123 determines the reference height of the visualization information based on height information, which is based on the height of the target object including the offset.

[0118] The presentation control unit 1123 adjusts the height of the visualization information to be included within a height range based on the target field of view. For example, the presentation control unit 1123 sets an offset to be included within the height range based on the target field of view. The presentation control unit 1123 determines the adjusted height of the visualization information as the reference height of the visualization information.

[0119] The presentation control unit 1123 determines information about the height of the visualization information at each location. The presentation control unit 1123 determines information about the height relative to a reference height of the visualization information, based on the irregularity (complexity) of the visualization information. For example, the presentation control unit 1123 determines the height of the visualization information at each location relative to the reference height of the visualization information, corresponding to the score calculated by the calculation unit 1122. The presentation control unit 1123 determines the height of the visualization information at each location based on both the reference height and the height based on the irregularity (complexity) of the visualization information. For example, the height of the visualization information at each location is determined by combining the reference height and the height based on the irregularity (complexity) of the visualization information. The presentation control unit 1123 determines information about the height of the visualization information based on the height information corresponding to the score of each of the multiple target objects included in the target range.

[0120] The presentation control unit 1123 determines information about the timing of displaying visual information to a target object. For example, the presentation control unit 1123 determines that when a user approaches a predetermined distance from the target object, visual information will be displayed to the target object based on information about the user's position.

[0121] • Presentation of Creation Unit 1124

[0122] The presentation creation unit 1124 has the function of controlling the visualization information to be displayed for the target object. The presentation creation unit 1124 controls the visualization information to be displayed for a target range that includes multiple target objects.

[0123] For example, the presentation creation unit 1124 controls the visualization information to be displayed for the target object based on information about the height of the visualization information determined by the presentation control unit 1123. Additionally, for example, the presentation creation unit 1124 controls the visualization information to be displayed for the target object based on information about timing determined by the presentation control unit 1123. Furthermore, for example, the presentation creation unit 1124 controls the visualization information to be displayed for the target object based on height information corresponding to the score calculated by the calculation unit 1122.

[0124] The presentation creation unit 1124 performs control for displaying visual information for a target object. For example, the presentation creation unit 1124 may perform control for displaying the height of the visual information determined by the presentation control unit 1123, that is, control for displaying visual information represented by a grid three-dimensional graphic. For example, the presentation creation unit 1124 may perform control for displaying visual information in which scores indicating the characteristics of the target object are represented by a grid three-dimensional graphic.

[0125] For example, the presentation creation unit 1124 can execute control to display visualization information represented by a three-dimensional path, where information about the height of the visualization information determined by the presentation control unit 1123 is represented by the visualization information. For example, the presentation creation unit 1124 can execute control to display visualization information represented by a three-dimensional path, where scores indicating relationships between multiple target objects are represented by the visualization information. Specifically, the presentation creation unit 1124 can execute control to display visualization information represented by a three-dimensional path, where scores indicating relationships between multiple target objects (e.g., interpersonal relationships in a town, product management in a warehouse, personnel allocation in a workplace such as an office, the liveliness of an event venue, etc.) are represented by the visualization information.

[0126] Figure 8 An example of visualization information indicated by a 3D path is shown. Virtual objects OB31 to OB34 are target objects included within the same target area. Figure 8 In the rendering, lines indicating paths originating from each virtual object indicate relationships between virtual objects. For example, virtual object OB31 is related to any of virtual objects OB32 to OB34. Therefore, the rendering creation unit 1124 can perform control to display virtual object OB31 as connected to any of virtual objects OB32 to OB34 via paths. Furthermore, the rendering creation unit 1124 can perform control to display virtual objects such that the stronger the relationship with another virtual object, the higher the density of the connecting paths. With this configuration, the rendering creation unit 1124 allows the user to easily grasp target objects that influence multiple target objects by the density of paths. Furthermore, when target objects are related to vegetation, the rendering creation unit 1124 allows the user to easily grasp plants that act as hubs influencing various locations.

[0127] For example, if the height difference of the height information based on the change in the visual information is greater than or equal to a predetermined threshold, the presentation creation unit 1124 can perform control to display the visual information in a specific mode. For example, the presentation creation unit 1124 compares the height information of the visual information based on information about the past target range with the height information of the visual information based on information about the current target range. Then, the presentation creation unit 1124 can perform control to dynamically display the visual information of locations where the height difference is equal to or greater than the predetermined threshold by using an animated view or the like.

[0128] Output unit 113

[0129] Output unit 113 has the function of outputting information about the visualization. Output unit 113 outputs information about the visualization based on the processing of processing unit 112. For example, output unit 113 outputs visualization information to a target object under the control of presentation creation unit 1124. Furthermore, output unit 113 outputs visualization information to the target object, for example, based on height information corresponding to the score calculated by calculation unit 1122.

[0130] Output unit 113 can output label information. For example, output unit 113 can output label information about a target object to display as visual information about the target object. Furthermore, for example, output unit 113 can output label information about a target object to display relative to the output visual information. Output unit 113 can output label information such that the label information is displayed at a location where the user can comfortably view the target object or visual information.

[0131] Output unit 113 provides information about the visualization information to be displayed for the target object. For example, output unit 113 provides control information for outputting the visualization information via communication unit 100. Note that output unit 113 can provide control information for outputting label information.

[0132] (1-3) Storage unit 120

[0133] Storage unit 120 is implemented using semiconductor memory elements such as random access memory (RAM) and flash memory drives, or storage devices such as hard disks or optical disks. Storage unit 120 has the function of storing data related to processing in information processing device 10.

[0134] Figure 9 An example of storage unit 120 is shown. Figure 9 The storage unit 120 shown stores information about a target range that includes multiple target objects. For example... Figure 9As shown, storage unit 120 may include items such as "target range ID", "target range", "target object ID" and "target object".

[0135] "Target Range ID" indicates identification information used to identify the target range. "Target Range" indicates information about the target range. Although... Figure 9 The example shown illustrates a scenario where conceptual information such as "Target Range #11" and "Target Range #12" is stored in "Target Range," but the actual coordinate information is stored in "Target Range." "Target Object ID" indicates identification information used to identify the target object. "Object" indicates information about the target object. Figure 9 The example shown illustrates a scenario where conceptual information such as "Target Object #11" and "Target Object #12" is stored in the "Target Object" folder. However, the information used to specify the target object (such as the target object's attributes and type) is actually stored in the "Target Object". The "Target Object" can also store information indicating its relationship to another target object.

[0136] (2) Terminal device 20

[0137] like Figure 6 As shown, the terminal device 20 includes a communication unit 200, a control unit 210, an output unit 220, and a sensor unit 230.

[0138] (2-1) Communication Unit 200

[0139] The communication unit 200 has the function of communicating with external devices. For example, when communicating with an external device, the communication unit 200 outputs information received from the external device to the control unit 210. Specifically, the communication unit 200 outputs information about visualization information received from the information processing device 10 to the control unit 210.

[0140] (2-2) Control Unit 210

[0141] The control unit 210 has the function of controlling the overall operation of the terminal device 20. For example, the control unit 210 performs processing to control the output of information about the visualization information.

[0142] (2-3) Output unit 220

[0143] The output unit 220 has the function of outputting information about the visualization. For example, the output unit 220 displays information about the visualization in AR representation.

[0144] (2-4) Sensor Unit 230

[0145] Sensor unit 230 has the function of acquiring sensor information measured by each measuring instrument. For example, sensor unit 230 acquires sensor information such as acceleration information, gyroscope information, GPS information, and geomagnetic information. Figure 6 As shown, the sensor unit 230 may include an acceleration sensor unit 231, a gyroscope sensor unit 232, a GPS receiver unit 233, and a geomagnetic sensor unit 234.

[0146] (3) Information providing device 30

[0147] like Figure 6 As shown, the information providing device 30 includes a communication unit 300, a control unit 310, and a storage unit 320.

[0148] (3-1) Communication Unit 300

[0149] The communication unit 300 has the function of communicating with external devices. For example, when communicating with an external device, the communication unit 300 outputs information received from the external device to the control unit 310. Specifically, the communication unit 300 outputs information received from the information processing device 10 to the control unit 310. For example, the communication unit 300 outputs information about the acquisition of information related to the target range to the control unit 310.

[0150] (3-2) Control Unit 310

[0151] The control unit 310 has the function of controlling the operation of the information providing device 30. For example, the control unit 310 transmits information about the target range to the information processing device 10 via the communication unit 300. For example, the control unit 310 transmits information about the target range obtained by accessing the storage unit 320 to the information processing device 10.

[0152] (3-3) Storage unit 320

[0153] Storage unit 320 stores information similar to that stored in storage unit 120. Therefore, a description of storage unit 320 is omitted.

[0154] <3.3. Information Processing System Processing>

[0155] The functions of the information processing system 1 according to the embodiment have been described above. Next, the processing of the information processing system 1 will be described.

[0156] (1) Processing 1 in information processing device 10: Control based on location information

[0157] Figure 10This is a flowchart illustrating the processing flow in the information processing apparatus 10 according to an embodiment. Specifically, the diagram shows the processing flow of the information processing apparatus 10 controlling the display of a virtual object based on location information about the user and information about a target range. First, the information processing apparatus 10 acquires location information about the user (S101). Next, the information processing apparatus 10 compares the location information about the user with the location information about the target range (S102). For example, the information processing apparatus 10 accesses the storage unit 120 and performs the comparison using the location information about the target range stored in the storage unit 120. Next, the information processing apparatus 10 determines whether a target range exists within a predetermined distance from the user's location (S103). If no target range exists within the predetermined distance from the user's location (S103; No), the information processing apparatus 10 terminates the information processing. If a target range exists within the predetermined distance from the user's location (Step S103; Yes), the information processing apparatus 10 determines whether to display a virtual object (S104).

[0158] If the information processing device 10 determines that the virtual object should not be displayed (S104; No), the process proceeds to step S106. Alternatively, if it has already been determined that the virtual object should be displayed (S104; Yes), the information processing device 10 displays the virtual object with an offset, based on information about the target range and the height of the real object (S105). Subsequently, the information processing device 10 determines whether to display visualization information (S106).

[0159] When it is determined that visualization information should not be displayed (S106; No), the information processing device 10 ends information processing. When it is determined that visualization information should be displayed (S106; Yes), the information processing device 10 calculates a score for the visualization information (S107). The information processing device 10 may provide information about the target range to an external information processing device and allow the external information processing device to calculate a score for the visualization information. Then, the information processing device 10 displays the visualization information based on the calculated score (S108). For example, the information processing device 10 displays the visualization information based on a score calculated by an external information processing device. When it is determined in step S104 that a virtual object should not be displayed, the information processing device 10 displays visualization information for a real object. When it is determined in step S104 that a virtual object should be displayed, the information processing device 10 displays visualization information for the virtual object to be displayed.

[0160] (2) Processing 2 in information processing device 10: Visual information height control 1

[0161] Figure 11This is a flowchart illustrating the processing flow in the information processing apparatus 10 according to an embodiment. Specifically, the diagram shows the process by which the information processing apparatus 10 controls the display height of visual information based on information about the height of the user's viewpoint, information about the distance between the user and the target area (e.g., relative horizontal distance), and information about the target area. First, the information processing apparatus 10 acquires information about the height of the user's viewpoint, information about the distance between the user and the target area, and information about the target area (S201). Next, the information processing apparatus 10 acquires information about target objects included in the target area (S202). As a specific example, if the target object is related to vegetation, the information processing apparatus 10 acquires information about the vegetation planted at each location within the target area.

[0162] The information processing device 10 calculates a score for the visualization information at each location based on information about the target object (S203). For example, if the target object is related to vegetation, the information processing device 10 calculates a score related to the vegetation planted at each location within the target area. Next, the information processing device 10 calculates height information corresponding to the score of the visualization information at each location based on the calculated scores (S204). For example, the information processing device 10 normalizes the height information corresponding to the score of the visualization information at each point based on the difference between the maximum and minimum values ​​of the scores calculated at each point. The information processing device 10 then calculates information about the height of the target object based on information about the target object (S205). For example, if the target object is related to vegetation, the information processing device 10 calculates information related to the height of the vegetation planted within the target area based on information related to the vegetation planted at each location within the target area. Then, the information processing device 10 displays the visualization information based on information about the normalized scores and information about the height of the target object (S206). For example, the information processing device 10 displays a grid-based 3D graphic in AR representation.

[0163] (3) Processing 3 in information processing device 10: Visual information height control 2

[0164] Figure 12This is a flowchart illustrating the processing flow in the information processing apparatus 10 according to an embodiment. Specifically, the flowchart illustrates the process by which the information processing apparatus 10 controls the display height of visual information based on an offset. First, the information processing apparatus 10 temporarily sets the height of the visual information to be greater than the height based on the height information of the real object (S301). For example, the information processing apparatus 10 temporarily sets the height of the visual information to be greater than the average height of the real object. Next, the information processing apparatus 10 determines whether the temporary set height of the visual information is included in the height range based on the target field of view (S302). When the temporary set height of the visual information is included in the height range based on the target field of view (S302; Yes), the information processing apparatus 10 determines the temporary set height of the visual information as the reference height of the visual information to be displayed (S303). When the temporary set height of the visual information is not included in the height range based on the target field of view (step S302; No), the information processing apparatus 10 sets the offset such that the temporary set height of the visual information is included in the height range based on the target field of view (S304). The information processing apparatus 10 then determines the reference height of the visual information to be displayed (S305).

[0165] (4) Processing 4 in information processing device 10: Visual information display control

[0166] Figure 13The diagram illustrates the processing flow in the information processing apparatus 10 according to an embodiment. Specifically, the diagram shows the processing flow of the information processing apparatus 10 displaying changes in the target range as visual information, represented in AR. First, the information processing apparatus 10 acquires information about the target range (S401). For example, the information processing apparatus 10 accesses the storage unit 120 and acquires the information about the target range stored in the storage unit 120. Next, the information processing apparatus 10 determines whether the acquired information about the target range is the same as the information about the target range displayed in the display view of the terminal device 20. When the acquired information about the target range is the same as the information about the target range displayed in the display view (step S402; Yes), the information processing apparatus 10 ends the information processing. When the acquired information about the target range is different from the information about the target range displayed in the display view (step S402; No), the information processing apparatus 10 calculates a score based on the visual information about the acquired information about the target range and a score based on the visual information about the target range displayed in the display view (S403). The information processing apparatus 10 then calculates the height difference corresponding to the calculated score (S404). Next, the information processing device 10 determines the display mode of the visualized information based on the calculated differences (S405). For example, the information processing device 10 may determine the changes at locations with large differences to be displayed in an animated view based on the magnitude of the calculated differences. Subsequently, the information processing device 10 displays the visualized information in AR representation based on the determined display mode (S406). Then, the information processing device 10 stores information about the target range displayed in the display view (S407).

[0167] Figure 13 The diagram illustrates a process where the information processing device 10 displays changes related to a target range using AR representation of changes in visual information. However, the information processing device 10 can also perform a process where changes related to a target range are represented using changes in virtual objects. In this case, in step S405, the information processing device 10 can determine the display mode of the virtual object based on the calculated differences. Furthermore, in step S406, the information processing device 10 can display the virtual object using AR representation based on the determined display mode.

[0168] <Processing Variations>

[0169] The embodiments of this disclosure have been described above. Next, variations of the processing in the embodiments of this disclosure will be described. Note that the variations of processing described below can be applied individually to the embodiments of this disclosure, or can be applied in combination to the embodiments of this disclosure. Furthermore, variations of processing can be applied instead of the configurations described in the embodiments of this disclosure, or variations of processing can be additionally applied to the configurations described in the embodiments of this disclosure.

[0170] (1) Normalization, standardization and truncation

[0171] Although the above implementation describes the case where the presentation control unit 1123 determines the height of the visualization information using the raw value of the height corresponding to the calculated score, the determination method is not limited to this example. The presentation control unit 1123 can determine the height of the visualization information by normalizing the height corresponding to the calculated score. For example, the presentation control unit 1123 can normalize the height corresponding to the calculated score to a value in the range of 0 to 1. The following formula (2) is an example of a normalization calculation formula.

[0172]

[0173] In the formula, E(x,y) represents the value at each location. E represents a numeric string. minE represents the minimum value of the numeric string. maxE represents the maximum value of the numeric string.

[0174] The presentation control unit 1123 can determine the height of the visualized information by standardizing the height corresponding to the calculated score. For example, the presentation control unit 1123 can standardize the height corresponding to the calculated score to a value in the range of 0 to 1. The following formula (3) shows an example of the calculation formula used for standardization.

[0175]

[0176] In the formula, E(x,y) represents the value at each location. E represents the numeric string. AveE represents the mean of the numeric string. σ represents the variance of the numeric string.

[0177] The presentation control unit 1123 can determine the height of the visualized information by limiting the height corresponding to the calculated score to a predetermined height range. For example, the presentation control unit 1123 can determine the height of the visualized information by truncating the height corresponding to the calculated score to a predetermined height range.

[0178] Figure 14 An example of a variation corresponding to the score is shown. Figure 14(A) shows visualization information based on the raw values ​​of the height corresponding to the score, KJ31. Figure 14 (B) shows a visualization of the values ​​obtained by highly normalizing the values ​​corresponding to the scores, KJ32. Figure 14 (C) shows a visualization of the values ​​obtained by highly standardizing the values ​​corresponding to the scores. Figure 14 (D) shows a visualization of the values ​​obtained by truncating the height corresponding to the score to a predetermined height range.

[0179] (2) Transmission rate of visualized information

[0180] The transmittance of the visualization information according to the implementation method can be predetermined. For example, the transmittance of the visualization information can be a fixed value. Furthermore, the transmittance of the visualization information can be determined, for example, based on the user's viewpoint. The information processing system 1 can determine the transmittance of the visualization information based on the height of the user's viewpoint, such that the higher the user's viewpoint, the higher the transmittance of the visualization information, and vice versa. With this configuration, the information processing system 1 can change the transmittance of the visualization information according to the user's state, such as sitting or standing, and thus can perform a display presentation that allows the user to view comfortably.

[0181] (3) Representation of scores

[0182] When the score according to the implementation method indicates the relationship of the target object, the score may be appropriately referred to as the "relationship score". Alternatively, when the target object is related to the Syneco farming method (registered trademark), the score according to the implementation method may be appropriately referred to as the "Syneco farming method score", "Syneco-score", or "Syneco farming method level". The score according to the implementation method is not limited to these examples and may be represented in any way.

[0183] (4 layers)

[0184] The above-described embodiment describes the case where the information processing device 10 determines the height of the displayed virtual object independently. Alternatively, the height of the displayed virtual object can be determined by layering the virtual objects according to their attributes (e.g., the moisture content of roots, leaves, and soil). For example, the information processing device 10 can layer the virtual objects such that the smaller the viewing angle of the virtual object to be displayed within the viewing angle, the higher the layer where the virtual object is to be displayed. With this configuration, the information processing device 10 can arrange virtual objects that are unlikely to unintentionally occupy the screen on the upper layer. Furthermore, the information processing device 10 can switch the layer of the height to be displayed based on the attributes of the virtual object.

[0185] (5) Viewpoint Height

[0186] The above embodiment describes the information processing device 10 acquiring information about the user's viewpoint height based on the user's standing position. Alternatively, information about the viewpoint height of a walking user can be acquired as information about the user's viewpoint height. Furthermore, the information processing device 10 can use a beacon to acquire information about the height above the ground.

[0187] (6) Slope

[0188] The above-described embodiment describes the case where the information processing device 10 displays a virtual object perpendicular to the target area based on information about the target area when the ground is a flat surface. Here, when the ground where the target area is located is a slope, the information processing device 10 can display the virtual object perpendicular to the target area based on information about the target area when the ground is a slope. With this configuration, even when the ground is a slope, the information processing device 10 can display the virtual object parallel to the slope.

[0189] (7) Granularity of visualized information

[0190] The information processing device 10 can adjust the granularity of the visualization information to be displayed. For example, the information processing device 10 can display visualization information based on the user's location information, such that the closer the user is to the target area, the larger the granularity of the visualization information to be displayed. In addition, the information processing device 10 can pre-determine the granularity of the visualization information corresponding to the user's location information, and switch the granularity of the visualization information to be displayed based on the user's location information.

[0191] (8) Selection based on user gaze

[0192] When multiple visual information items to be displayed exist within the display viewpoint, the information processing device 10 can determine the visual information of interest to the user based on the user's line of sight. The information processing device 10 can then determine the visual information of interest to be dynamically displayed. For example, the information processing device 10 can determine the visual information closest to the user's line of sight as the visual information to be dynamically displayed. Furthermore, for example, when the user's viewpoint is lowered, such as when the user is seated, the information processing device 10 can perform specific processing on visual information other than the visual information of interest to the user. For example, the information processing device 10 can perform specific processing such as transparent display of visual information or separation of the position of visual information.

[0193] (9) Other application examples

[0194] The above-described embodiments can be applied to the visualization of livestock data, such as shellfish farming. This also allows the information processing device 10 to apply the technology to the visualization of livestock data, such as underwater data that is not visible to the user.

[0195] <<4. Hardware Configuration Examples>>

[0196] Finally, refer to Figure 15 An example of the hardware configuration of an information processing apparatus according to an embodiment is described. Figure 15 This is a block diagram illustrating an example hardware configuration of an information processing apparatus according to an embodiment. Note, for example, Figure 15 The information processing device 900 shown can achieve Figure 6 The information processing apparatus 10, terminal device 20, and information providing device 30 shown are described. The information processing implemented by the information processing apparatus 10, terminal device 20, and information providing device 30 according to the embodiment is achieved in cooperation with the software and hardware described below.

[0197] like Figure 15 As shown, the information processing device 900 includes a central processing unit (CPU) 901, a read-only memory (ROM) 902, and a random access memory (RAM) 903. Furthermore, the information processing device 900 includes a host bus 904a, a bridge 904, an external bus 904b, an interface 905, an input device 906, an output device 907, a storage device 908, a driver 909, a connection port 910, and a communication device 911. Note that the hardware configuration shown here is an example, and some components may be omitted. In addition, the hardware configuration may also include components other than those shown here.

[0198] The CPU 901 functions as, for example, an arithmetic processing device or a control device, and controls all or part of the operation of each component based on various programs recorded in the ROM 902, RAM 903, or storage device 908. The ROM 902 is a device for storing programs loaded by the CPU 901, data used for calculation, etc. The RAM 903 temporarily or permanently stores, for example, programs loaded by the CPU 901, various parameters that change appropriately during program execution, etc. These components are interconnected via a host bus 904a, including a CPU bus, etc. The CPU 901, ROM 902, and RAM 903 can, for example, cooperate with software to implement a reference... Figure 6 The functions of control units 110, 210 and 310 are described.

[0199] The CPU 901, ROM 902, and RAM 903 are interconnected, for example, via a host bus 904a capable of high-speed data transfer. On the other hand, the host bus 904a is connected, for example, via a bridge 904 to an external bus 904b with a relatively low data transfer speed. Furthermore, the external bus 904b is connected to various components via an interface 905.

[0200] Input device 906 is implemented by a device for a lister to input information, such as a mouse, keyboard, touch panel, button, microphone, switch, and joystick. Alternatively, input device 906 can be a remote control device using infrared or other radio waves, or an external connection device such as a mobile phone or PDA that supports the operation of information processing device 900. Furthermore, input device 906 may include, for example, an input control circuit that generates an input signal based on the input information used by the aforementioned input device and outputs the input signal to CPU 901. By operating input device 906, the administrator of information processing device 900 can input various data into information processing device 900 and issue instructions regarding processing operations.

[0201] Furthermore, the input device 906 can be formed by a device for detecting the user's location. For example, the input device 906 may include various sensors, such as image sensors (e.g., camera devices), depth sensors (e.g., stereo camera devices), accelerometers, gyroscopes, geomagnetic sensors, optical sensors, sound sensors, distance measurement sensors (e.g., time-of-flight (ToF) sensors), and force sensors. Additionally, the input device 906 can acquire information about the information processing device 900's own state, such as the attitude and speed of movement of the information processing device 900, and information about the surrounding space of the information processing device 900, such as the brightness and noise around the information processing device 900. Furthermore, the input device 906 may include a Global Navigation Satellite System (GNSS) module that receives GNSS signals from GNSS satellites (e.g., GPS signals from Global Positioning System (GPS) satellites) and measures location information including the device's latitude, longitude, and altitude. Furthermore, regarding location information, the input device 906 can detect its location via Wi-Fi (registered trademark), transmission and reception using mobile phones, PHS, smartphones, near-field communication, etc. For example, the input device 906 can realize a reference. Figure 6 The function of the sensor unit 230 is described.

[0202] Output device 907 is formed of a device capable of notifying a user of acquired information visually or audibly. Examples of such devices include display devices such as CRT display devices, liquid crystal display devices, plasma display devices, EL display devices, laser projectors, LED projectors, and lamp display devices; audio output devices such as speakers and headphones; and printer devices. For example, output device 907 outputs the results obtained from various processes performed by information processing device 900. Specifically, the display device visually displays the results obtained from various processes performed by information processing device 900 in various formats such as text, images, tables, and graphics. The audio output device converts an audio signal, consisting of reproduced audio data, sound data, etc., into an analog signal and outputs the signal audibly. Output device 907 can realize, for example, reference... Figure 6 The function of the output unit 220 is described.

[0203] Storage device 908 is an example data storage device formed as a storage unit in information processing apparatus 900. Storage device 908 is implemented using, for example, magnetic storage devices such as HDDs, semiconductor memory devices, optical storage devices, and opto-magnetic storage devices. Storage device 908 may include a storage medium, a recording device for recording data on the storage medium, a reading device for reading data from the storage medium, and a deletion device for deleting data recorded on the storage medium. This storage device 908 stores programs executed by CPU 901, various data, and various data acquired from external sources. For example, storage device 908 can implement reference... Figure 6 The function of the storage unit 120 is described.

[0204] Drive 909 is a reader / writer for storage media and is either built into or externally connected to information processing device 900. Drive 909 reads information recorded on removable storage media such as mounted disks, optical disks, magneto-optical disks, or semiconductor memories, and outputs the read information to RAM 903. Drive 909 can also write information to removable storage media.

[0205] Connection port 910 is, for example, a port for connecting external connection devices, such as a Universal Serial Bus (USB) port, an IEEE 1394 port, a Small Computer System Interface (SCSI) port, an RS-232C port, or an optical audio terminal.

[0206] Communication device 911 is, for example, a communication interface formed by a communication device for connecting to network 920. Communication device 911 is, for example, a communication card for wired or wireless local area networks (LANs), Long Term Evolution (LTE), Bluetooth (registered trademark), Wireless USB (WUSB), etc. Furthermore, communication device 911 can be an optical communication router, an Asymmetric Digital Subscriber Line (ADSL) router, a modem for various communications, etc. Communication device 911 can exchange signals via the Internet and other communication devices according to predetermined protocols such as TCP / IP. Communication device 911 can implement, for example, [referring to...] Figure 6 The functions of communication unit 100, communication unit 200 and communication unit 300 are described.

[0207] Network 920 is a wired or wireless transmission path for transmitting information from devices connected to network 920. For example, network 920 may include public networks such as the Internet, telephone networks, and satellite communication networks, or various local area networks (LANs), including Ethernet (registered trademark), wide area networks (WANs), etc. Furthermore, network 920 may include private networks such as Internet Protocol-Virtual Private Networks (IP-VPNs).

[0208] Examples of hardware configurations capable of implementing the functions of the information processing apparatus 900 according to the embodiments have been described above. Each of the above-described components can be implemented using general-purpose components, or can be implemented using hardware devices dedicated to the functions of each component. Therefore, the hardware configuration to be used can be appropriately changed depending on the level of technology at the time of performing the embodiments.

[0209] <<5. Summary>>

[0210] As described above, the information processing apparatus 10 according to the embodiment performs processing for displaying visual information for the target object based on height information corresponding to a score indicating the characteristics of the target object. With this configuration, the information processing apparatus 10 enables the user to accurately grasp the association between the target object and the visual information, thereby further enhancing usability.

[0211] Furthermore, the information processing device 10 performs processing to display visualization information in which the complexity of the target objects has been converted into height information. With this configuration, the information processing device 10 can display conceptual information associated with each target object at the height of the visualization information, allowing the conceptual information to adapt to the experience accompanying user behavior. This enables the information processing device 10 to facilitate the user's understanding of the target objects. Moreover, the information processing device 10 can visualize the relationships between target objects that are difficult to determine solely through numerical values, thereby further facilitating the user's understanding of the target objects.

[0212] Furthermore, the information processing device 10 adjusts the height of the visual information to be displayed to a height that does not interfere with the display of the target object. For example, the information processing device 10 derives the height of the visual information above the ground based on the user's viewpoint height, the relative horizontal distance between the user and the target area, and the height of the target object included in the target area. With this configuration, the information processing device 10 can display the visual information without completely hiding the target object. Specifically, the information processing device 10 can display the visual information and the target object without overlapping. Therefore, the information processing device 10 can easily associate the target object with the visual information indicating conceptual information.

[0213] Furthermore, the information processing device 10 can retrospectively display visualizations based on information about the past target range. Additionally, the information processing device 10 can perform processing to dynamically display changes in information about the target range by comparing visualizations based on information about the past target range with visualizations based on information about the current target range. With this configuration, the information processing device 10 can dynamically visualize changes in the target range using animated views or the like. Therefore, the information processing device 10 allows users to easily grasp changes in the target range caused by user actions such as harvesting or sowing.

[0214] This enables the provision of novel and improved information processing devices, methods, and procedures that can further enhance usability.

[0215] Preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings. However, the technical scope of the present disclosure is not limited to these examples. Obviously, those skilled in the art can make various changes or modifications within the scope of the technical concept described in the claims, and such changes or modifications should naturally be understood to fall within the technical scope of the present disclosure.

[0216] For example, each device described in this specification can be implemented as a standalone device, or some or all of the devices can be implemented as separate devices. For example, Figure 6 The information processing device 10, terminal device 20, and information providing device 30 shown can be implemented as independent devices. Alternatively, for example, the device can be implemented as a server device connected to the information processing device 10, terminal device 20, and information providing device 30 via a network or the like. Alternatively, the functions of the control unit 110 included in the information processing device 10 can be included in the server device connected via a network or the like.

[0217] Furthermore, the series of processes to be performed by the various devices described herein can be implemented using software, hardware, or a combination of both. The programs constituting the software are pre-stored in a recording medium (non-transitory medium) located, for example, inside or outside each device. Each program is then read into RAM, for example, when executed by a computer, and executed by a processor such as a CPU.

[0218] Furthermore, the processes described using flowcharts in this specification do not necessarily have to be executed in the order shown. Some processing steps can be executed in parallel. In addition, additional processing steps may be used, and some processing steps may be omitted.

[0219] Furthermore, the effects described in this specification are merely illustrative or exemplary and not restrictive. That is, in addition to or instead of the effects described above, the technology according to this disclosure may exhibit other effects that will be apparent to those skilled in the art as described in this specification.

[0220] Note that the following configurations also fall within the technical scope of this disclosure. (1)

[0222] An information processing device, comprising:

[0223] The calculation unit calculates a score indicating the characteristics of the target object; and

[0224] The presentation control unit performs processing for displaying visual information for the target object based on height information corresponding to the score calculated by the calculation unit. (2)

[0226] According to the information processing device described in (1),

[0227] The presentation control unit performs processing for displaying the visualization information based on a reference height determined by the visualization information based on the height information and the height information based on the target object. (3)

[0229] According to the information processing device described in (2),

[0230] The presentation control unit performs processing for displaying the visualization information based on height information that is based on the height of the target object, including the maximum, average, mode, or minimum height of the target object. (4)

[0232] According to the information processing device described in (2) or (3),

[0233] The presentation control unit performs processing for displaying the visualization information based on height information based on the height of the target object. The height information includes an offset determined based on the height of the target object and the height of the visualization information to be displayed. (5)

[0235] The information processing apparatus according to any one of (1) to (4),

[0236] Specifically, when the visualization information to be displayed falls within the viewing angle determined based on the user's viewpoint height and the relative horizontal distance between the user and the target object, the presentation control unit performs processing for displaying the visualization information. (6)

[0238] According to the information processing device described in (5),

[0239] Specifically, when the visual information to be displayed falls within the viewpoint of a display viewpoint determined based on information related to the angle of the user's field of vision, the presentation control unit performs processing for displaying the visual information. (7)

[0241] The information processing apparatus according to any one of (1) to (6),

[0242] The calculation unit calculates the score for each of the multiple target objects, and

[0243] The presentation control unit performs processing for displaying the visualization information based on the height information corresponding to the score of each of the plurality of target objects. (8)

[0245] The information processing apparatus according to any one of (1) to (7),

[0246] Specifically, when the height difference of the height information indicating the change of the visualization information is equal to or greater than a predetermined threshold, the presentation control unit performs processing for dynamically displaying the visualization information. (9)

[0248] The information processing apparatus according to any one of (1) to (8),

[0249] The presentation control unit performs processing to display the visualization information in which the score is represented by a grid three-dimensional graphic. (10)

[0251] The information processing apparatus according to any one of (1) to (9),

[0252] The calculation unit calculates a score indicating the characteristics of the target object related to vegetation. (11)

[0254] The information processing apparatus according to any one of (1) to (10),

[0255] The calculation unit calculates a score indicating the relationship between multiple target objects, and

[0256] The presentation control unit performs processing to display visualization information that visualizes the relationship between the multiple target objects using a three-dimensional path, based on a score calculated by the computing unit that indicates the relationship between the multiple target objects. (12)

[0258] According to the information processing device described in (11),

[0259] The presentation control unit performs processing to display the visualization information that visualizes interpersonal relationships in a town, product management in a warehouse, personnel allocation in a workplace, or the liveliness of a venue. (13)

[0261] The information processing apparatus according to any one of (1) to (12),

[0262] The presentation control unit performs processing for displaying the visual information in augmented reality (AR) representation. (14)

[0264] An information processing method executed by a computer, the method comprising:

[0265] The calculation steps involve calculating scores that indicate the characteristics of the target object; and

[0266] The presentation control step, based on the height information corresponding to the score calculated through the calculation step, performs processing for displaying visualization information for the target object. (15)

[0268] An information processing program that causes a computer to execute:

[0269] The calculation process, which calculates a score indicating the characteristics of the target object; and

[0270] The presentation control process, based on height information corresponding to the score calculated through the calculation process, performs processing for displaying visual information for the target object.

[0271] List of reference numerals

[0272] 1. Information Processing System

[0273] 10. Information processing device

[0274] 20 Terminal devices

[0275] 30 Information providing devices

[0276] 100 communication units

[0277] 110 Control Unit

[0278] 111 Acquisition Unit

[0279] 112 Processing Units

[0280] 1121 Position Control Unit

[0281] 1122 Computing Unit

[0282] 1123 Presentation Control Unit

[0283] 1124 Presentation of the creation unit

[0284] 113 Output Unit

[0285] 120 storage units

[0286] 200 communication units

[0287] 210 Control Unit

[0288] 220 Output Unit

[0289] 230 sensor units

[0290] 300 communication units

[0291] 310 Control Unit

[0292] 320 storage units

Claims

1. An information processing apparatus, comprising: A calculation unit that calculates a score indicating the characteristics of the target object; as well as The presentation control unit performs processing for displaying visual information for the target object based on height information corresponding to the score calculated by the calculation unit. Specifically, when the visualization information to be displayed falls within the viewing angle determined based on information related to the user's viewpoint height, the relative horizontal distance between the user and the target object, and the angle of the user's field of vision, the presentation control unit performs processing for displaying the visualization information.

2. The information processing device according to claim 1, in, The presentation control unit performs processing for displaying the visualization information based on a reference height determined by the visualization information based on the height information and the height information based on the target object.

3. The information processing device according to claim 2, in, The presentation control unit performs processing for displaying the visualization information based on height information that is based on the height of the target object, including the maximum, average, mode, or minimum height of the target object.

4. The information processing device according to claim 2, in, The presentation control unit performs processing for displaying the visualization information based on height information based on the height of the target object, the height information including an offset determined based on the height of the target object and the height of the visualization information to be displayed.

5. The information processing device according to claim 1, in, The computing unit calculates the score for each of the multiple target objects, and The presentation control unit performs processing for displaying the visualization information based on the height information corresponding to the score of each of the plurality of target objects.

6. The information processing apparatus according to claim 1, in, When the height difference of the height information indicating the change of the visualization information is equal to or greater than a predetermined threshold, the presentation control unit performs processing for dynamically displaying the visualization information.

7. The information processing apparatus according to claim 1, in, The presentation control unit performs processing to display the visualization information, in which the score is represented by a grid of three-dimensional graphics.

8. The information processing apparatus according to claim 1, in, The computing unit calculates a score indicating the characteristics of the target object related to vegetation.

9. The information processing apparatus according to claim 1, in, The computing unit calculates a score indicating the relationship between multiple target objects, and The presentation control unit performs processing to display visualization information that visualizes the relationship between the multiple target objects using a three-dimensional path, based on a score calculated by the computing unit that indicates the relationship between the multiple target objects.

10. The information processing apparatus according to claim 9, in, The presentation control unit performs processing to display the visualization information that visualizes interpersonal relationships in a town, product management in a warehouse, personnel allocation in a workplace, or the liveliness of a venue.

11. The information processing apparatus according to claim 1, in, The presentation control unit performs processing for displaying the visual information in an augmented reality representation.

12. An information processing method executed by a computer, the method comprising: The calculation steps involve calculating a score that indicates the characteristics of the target object; as well as The presentation control step, based on height information corresponding to the score calculated through the calculation step, performs processing for displaying visual information for the target object. Specifically, if the visualization information to be displayed falls within the viewpoint determined based on the user's viewpoint height, the relative horizontal distance between the user and the target object, and the angle related to the user's field of vision, then processing for displaying the visualization information is performed.

13. An information processing program that causes a computer to execute: The calculation process, which calculates a score indicating the characteristics of the target object; and The presentation control process, based on height information corresponding to the score calculated through the calculation process, performs processing for displaying visual information for the target object. in, If the visualization information to be displayed falls within the viewpoint determined based on the user's viewpoint height, the relative horizontal distance between the user and the target object, and the angle related to the user's field of vision, then processing for displaying the visualization information is performed.

14. A recording medium containing an information processing program, said information processing program causing a computer to execute: The calculation process, which calculates a score indicating the characteristics of the target object; and The presentation control process, based on height information corresponding to the score calculated through the calculation process, performs processing for displaying visual information for the target object. in, If the visualization information to be displayed falls within the viewpoint determined based on the user's viewpoint height, the relative horizontal distance between the user and the target object, and the angle related to the user's field of vision, then processing for displaying the visualization information is performed.

Citation Information

Patent Citations

  • Information processing device and information processing method

    WO2017061281A1

  • Information processing device and information processing method

    US20180271027A1

  • Placement of augmented reality objects using a guide marker

    US20190197725A1

  • Augmented reality for plant stand management

    US20190311197A1