Information processing device and program

Abstract

To provide an information processing device that can appropriately simulate the camera's installation position, taking into account how the camera appears from the user's perspective when using the space. [Solution] The information processing device 10 according to this disclosure includes a simulation unit 12 that simulates the camera's installation status and shooting status using a virtual space that includes a camera that takes pictures in the virtual space and objects that can move in the virtual space, and a display control unit 13 that displays a display screen that includes a camera view that displays a captured image based on the camera's viewpoint and an object view that displays a field of view based on the object's viewpoint.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus and a program.

Background Art

[0002] A technology for simulating the shooting situation of a camera installed in a virtual space using the virtual space is known. As a related technology, Patent Document 1 discloses a technology for acquiring point cloud data of a monitoring area, virtually installing a monitoring camera, and generating a virtual monitoring image according to the installation position of the camera.

Prior Art Document

Patent Document

[0003]

Patent Document 1

Summary of Invention

Problem to be Solved by the Invention

[0004] By using the technology as described above, an installer of a camera can easily know the shooting situation of the camera without installing the camera in a real space. However, in a real space, it may be desirable to install the camera in consideration of how the camera looks from the viewpoint of a user who uses the space. For example, a surveillance camera installed in a bank can contribute to crime prevention by being installed at a position easily visible to customers. Also, for example, a surveillance camera installed in a cafe can be installed at a position that is not easily visible to customers, so that customers can spend their time relaxing.

[0005] In view of the above-mentioned problems, an object of the present disclosure is to provide an information processing apparatus and a program capable of appropriately simulating the installation position of a camera in consideration of how the camera looks from the viewpoint of a user who uses a space.

Means for Solving the Problem

[0006] The information processing device relating to this disclosure is A simulation unit that simulates the camera's installation and shooting conditions using a virtual space that includes a camera that takes pictures within the virtual space and objects that can move within the virtual space. The system includes a display control unit that displays a display screen including a camera view that displays a captured image based on the viewpoint of the camera, and an object view that displays a field of view based on the viewpoint of the object.

[0007] The program related to this disclosure is: A simulation step in which the camera's installation and shooting conditions are simulated using a virtual space that includes a camera for taking pictures within the virtual space and objects that can move within the virtual space, The system causes a computer to perform a display control step that displays a display screen including a camera view that displays a captured image based on the viewpoint of the camera, and an object view that displays a field of view based on the viewpoint of the object. [Effects of the Invention]

[0008] The information processing device and program described herein can appropriately simulate the camera's installation position, taking into account how the camera appears from the perspective of a user utilizing the space. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 is a block diagram showing the configuration of the information processing device according to this disclosure. [Figure 2] Figure 2 shows an example of a display screen including the camera view and the user view. [Figure 3] Figure 3 shows an example of a display screen including the camera view and user view in a location different from the one shown in Figure 2. [Figure 4] Figure 4 is a flowchart showing the processing flow performed by the information processing device. [Modes for carrying out the invention]

[0010] Embodiments of the present disclosure will be described in detail below with reference to the drawings. In each drawing, the same or corresponding elements are denoted by the same reference numerals. For clarity of explanation, redundant explanations will be omitted where necessary.

[0011] (Configuration of the information processing device 10) The information processing device 10 according to this disclosure will be described with reference to Figure 1. Figure 1 is a block diagram showing the configuration of the information processing device 10. The information processing device 10 comprises a virtual space generation unit 11, a simulation unit 12, a display control unit 13, a display unit 14, and an input unit 15. The information processing device 10 is a device that performs a simulation regarding the installation position of a camera in a virtual space and executes predetermined display control.

[0012] The information processing device 10 includes a processor, memory, and storage device (not shown). The storage device stores a computer program on which the processing described herein is implemented. The processor can load the computer program from the storage device into memory and execute the computer program. In this way, the processor realizes the functions of the virtual space generation unit 11, the simulation unit 12, and the display control unit 13.

[0013] Alternatively, the virtual space generation unit 11, the simulation unit 12, and the display control unit 13 may each be implemented with dedicated hardware. Furthermore, some or all of each component may be implemented by general-purpose or dedicated circuits, processors, etc., or combinations thereof. These may be configured by a single chip or by multiple chips connected via a bus. Some or all of each component may be implemented by a combination of the aforementioned circuits, etc., and programs. Additionally, a CPU (Central Processing Unit), GPU (Graphics Processing Unit), FPGA (Field-Programmable Gate Array), quantum processor (quantum computer control chip), etc., can be used as the processor.

[0014] Furthermore, if some or all of the components of the information processing device 10 are realized by multiple information processing devices or circuits, these multiple information processing devices or circuits may be centrally located or distributed. For example, the information processing devices or circuits may be realized in a form in which each is connected via a communication network, such as a client-server system or a cloud computing system. Also, the functions of the information processing device 10 may be provided in SaaS (Software as a Service) format.

[0015] The virtual space generation unit 11 generates a virtual space. The virtual space may be a simulation space that reproduces a real environment. For example, the virtual space generation unit 11 generates a virtual space that reproduces a space such as a store, office, public facility, or residence. However, it is not limited to these, and the virtual space generation unit 11 may generate a virtual space that reproduces any space that is the subject of the simulation. For example, the virtual space generation unit 11 generates a metaverse space.

[0016] The virtual space generation unit 11 may generate a virtual space using any method. For example, the virtual space generation unit 11 generates a virtual space by using a virtual space construction technique that uses a three-dimensional model. The virtual space generation unit 11 reproduces a real space based on, for example, architectural data or point cloud data of a space to be simulated.

[0017] The virtual space generation unit 11 may also generate a virtual space based on captured data of real space. The virtual space generation unit 11 may acquire captured data from a person who performs simulation (hereinafter referred to as an "operator"). The operator is, for example, a person who is considering installing a camera in real space.

[0018] Specifically, first, the operator uses an imaging device such as a smartphone to capture an image of the space to be simulated. The operator inputs the captured data obtained through imaging to the virtual space generation unit 11. The virtual space generation unit 11 accepts the input of the captured data. The virtual space generation unit 11 generates a virtual space reproduced at an actual scale based on the captured data. The virtual space generation unit 11 may distribute the URL of a webpage accessible to the generated virtual space to the operator or other parties.

[0019] In addition, the virtual space generation unit 11 generates a virtual space that includes at least a camera for imaging the interior of the virtual space and an object movable within the virtual space. The virtual space generation unit 11 may include elements other than these in the virtual space. Furthermore, the virtual space generation unit 11 may arrange a plurality of cameras and a plurality of objects respectively in the virtual space.

[0020] The camera is installed at a predetermined position in the virtual space and photographs the interior of the virtual space from the installation position. The camera may be a virtual reproduction of a commercially available camera distributed in real space, or may be an arbitrarily set virtual camera. For example, the virtual space generation unit 11 constructs a virtual camera based on the specifications of a surveillance camera, a security camera, a web camera, or an industrial camera used in real space. The camera may include a single-focus lens or a zoom lens. The virtual space generation unit 11 may arbitrarily set imaging conditions such as the angle of view, focal length, or distortion correction of the camera.

[0021] An object is a component arranged in the virtual space. The object is configured to be movable within the virtual space. The object may be configured to move based on an operator's input received by the input unit 15. For example, the virtual space generation unit 11 configures an object as a virtual model of a person, an animal, a vehicle, or the like.

[0022] The following description uses an example in which an object is configured as a human-shaped avatar. The avatar corresponds to a virtual user that moves within the virtual space. In the following description, the virtual user is simply referred to as a "user". For example, when the virtual space is a space that reproduces a store or a public facility, the user simulates a customer or a visitor who uses the space.

[0023] The user may be configured to be able to freely move and rotate within the virtual space based on an input received by the input unit 15. Further, the user may be configured to be able to freely change the position and posture of body parts based on the input.

[0024] Furthermore, users may be able to arbitrarily set attributes such as height, build, or movement speed. For example, if the user's height is different, the position of the user's viewpoint will also be different. The virtual space generation unit 11 may accept the user's height specification from the operator via the input unit 15. The virtual space generation unit 11 places a user with the specified height in the virtual space. With these settings, the operator can place a user in the virtual space according to the simulation conditions. The virtual space generation unit 11 outputs the specified user's height and the user's initial position to the simulation unit 12.

[0025] The simulation unit 12 simulates the camera's installation and shooting conditions using a virtual space that includes a camera that takes pictures within the virtual space and objects that can move within the virtual space. Here, the simulation unit 12 performs the simulation using the virtual space generated by the virtual space generation unit 11.

[0026] The simulation unit 12 performs a simulation using a virtual space based on the operator's input received by the input unit 15. The operator's input received by the input unit 15 may include operation information for manipulating components within the virtual space. Operation information indicates the content of the operation that the operator intends to perform on the components within the virtual space. Operation information may include, for example, information indicating a change in the camera's installation position, a change in the camera's orientation, the user's movement, or a change in the user's viewpoint. By inputting the camera's installation position, the user's viewpoint, etc., the operator can set the state of the camera and user within the virtual space to a desired state and perform the simulation.

[0027] Furthermore, operation information may not be limited to the camera and the user, but may also indicate operations on a free viewpoint within the virtual space. For example, operation information may indicate a change in the position of the free viewpoint. Moreover, operation information may indicate an operation to change the display mode of the display screen viewed by the operator. Specific examples of operation information will be described later.

[0028] The simulation unit 12 displays the simulation results on the display unit 14. By viewing the display unit 14 with the simulation results, the operator can easily understand the camera and user status in the virtual space.

[0029] Furthermore, the simulation unit 12 identifies the field of view from the user's viewpoint based on the position of the user's viewpoint. The user's field of view indicates the area visible from the user's viewpoint. The field of view is the area of ​​space contained within a certain angle relative to the user's viewpoint. The simulation unit 12 may identify the position of the user's viewpoint based on at least one of the user's height, posture, and gaze direction information. The simulation unit 12 identifies the range that falls within the user's field of vision and identifies this range as the user's field of view.

[0030] For example, the simulation unit 12 first obtains the user's height and initial position from the virtual space generation unit 11 and determines the field of view in the initial state. For example, based on the acquired height and initial position information, the simulation unit 12 determines the range visible from the user's viewpoint at the initial position as the field of view at the initial position. The field of view at the initial position indicates the user's field of view immediately after the user is placed in the virtual space. The simulation unit 12 may also determine the field of view by taking into account the presence of obstacles in the virtual space.

[0031] The simulation unit 12 can dynamically acquire changes in the user's position and posture based on operation information received via the input unit 15. The simulation unit 12 updates the user's field of view based on the user's field of view at the initial position and the operation information. The simulation unit 12 updates the user's field of view in real time in response to changes in the user's state (for example, changes in the user's position and posture). As a result, even if the user moves or changes the orientation of their body, the simulation unit 12 reflects the change in the field of view relative to the initial position in the current field of view. This allows the operator to accurately understand the user's field of view in the virtual space.

[0032] Furthermore, the simulation unit 12 may classify the user's field of view into multiple field of view regions based on the user's viewpoint. The field of view regions represent differences in the user's ability to identify objects within the field of view.

[0033] The field of view consists of the central field of view, the effective field of view, and the peripheral field of view. The central field of view is the range in which a user can accurately distinguish the letters, colors, and shapes of objects, and is approximately 1 degree from the center of their line of sight. The effective field of view is the range in which a user can recognize objects with near-clearness, and is approximately 4 to 20 degrees from the center of their line of sight. The peripheral field of view is the range in which a user can recognize the shape and movement of objects, and is approximately 20 to 100 degrees from the center of their line of sight. However, it is difficult for users to distinguish detailed information in the peripheral field of view.

[0034] The simulation unit 12 classifies the identified field of view into central field of view, effective field of view, and peripheral field of view, and outputs the classification result as field of view area information to the display control unit 13. Field of view area information indicates which field of view area each region within the field of view belongs to. Specifically, the field of view area information indicates which part of the user's field of view corresponds to the central field of view, effective field of view, or peripheral field of view.

[0035] The simulation unit 12 identifies the spatial location corresponding to each field of view and determines which range in the virtual space each field of view corresponds to. Based on this information, the simulation unit 12 determines which field of view each camera within the user's field of view belongs to. For example, the simulation unit 12 outputs information such as "there is one camera in the effective field of view and one in the peripheral field of view" as field of view information.

[0036] Furthermore, the simulation unit 12 may identify recommended camera placement locations based on field-of-view information and propose these recommended locations to the operator. The recommended placement locations are positions within the virtual space where camera placement is recommended. The simulation unit 12 may propose multiple recommended placement locations.

[0037] For example, in financial institutions such as banks, it is desirable to install cameras in locations easily visible to users for security purposes. Similarly, in restaurants such as cafes, it is desirable to install cameras in inconspicuous locations that are difficult for users to notice. Furthermore, the camera placement should consider not only the perspective of customers but also that of the store staff. For instance, it may be desirable to install cameras in a location visible to kitchen staff.

[0038] For example, if the camera is to be conspicuous, the simulation unit 12 suggests a recommended installation location that is included in the user's central field of view or effective field of view. If the camera is not to be conspicuous, the simulation unit 12 suggests a recommended installation location that is outside the user's peripheral field of view or field of view. Based on the conditions specified by the operator, the simulation unit 12 identifies a location where the user can easily recognize the camera or a location where the user cannot easily recognize it. The simulation unit 12 identifies this location as a recommended installation location and proposes the recommended installation location to the operator.

[0039] For example, the simulation unit 12 may set up multiple users with different viewpoints in the virtual space and determine the recommended installation location using the field of view of these multiple users. For example, the simulation unit 12 may set up multiple users with different heights. This allows the simulation unit 12 to determine the field of view range of each of the multiple users with different viewpoint heights.

[0040] The simulation unit 12 calculates the degree of overlap, which indicates the extent to which a predetermined installation location is included in the field of view of multiple users. The simulation unit 12 may obtain the predetermined installation location via the input unit 15. The predetermined installation location may be, for example, the ceiling, wall, column, counter, or floor in the virtual space, but is not limited to these. The simulation unit 12 calculates the degree of overlap such that the larger the area where the field of view of multiple users overlaps, or the larger the number of overlapping users, the higher the degree of overlap. The degree of overlap may be expressed using numerical values ​​or multiple levels.

[0041] The simulation unit 12 may calculate the degree of overlap using the field of view area instead of the field of view range. In this case, the simulation unit 12 calculates the degree of overlap as the extent to which a predetermined installation location is included in the field of view area of ​​each of multiple users. The simulation unit 12 may calculate the degree of overlap for the central field of view, the effective field of view, and the peripheral field of view. Alternatively, the simulation unit 12 may calculate the degree of overlap by combining two of the central field of view, the effective field of view, and the peripheral field of view. For example, the simulation unit 12 may calculate the degree of overlap for the combined area of ​​the effective field of view and the peripheral field of view.

[0042] The simulation unit 12 may score the degree of overlap. For example, the simulation unit 12 may set different weights for each field of view and score the degree of overlap using the set weights. In this case, the simulation unit 12 scores the degree of overlap using a numerical value corresponding to the weight set for each field of view, based on whether the predetermined installation position is included in each user's central field of view, effective field of view, or peripheral field of view.

[0043] The simulation unit 12 analyzes the field of view or field of view area and identifies locations with a high or low degree of overlap in the field of view. For example, the simulation unit 12 identifies locations where the number of users whose field of view includes the camera is greater than a predetermined value as locations where the camera is likely to be recognized. The simulation unit 12 then proposes locations with a large number of users whose field of view overlaps as recommended installation locations in places like banks.

[0044] Furthermore, for example, the simulation unit 12 identifies locations where the number of users whose field of view includes the camera is less than a predetermined value as locations where the camera is less likely to be recognized. The simulation unit 12 then proposes these locations with fewer overlapping users in the field of view as recommended installation locations in restaurants and other establishments.

[0045] For example, the simulation unit 12 may present a message to the operator suggesting a recommended installation location, such as, "The recommended installation location for surveillance cameras at X Bank is the ceiling directly in front of the entrance." The simulation unit 12 may also display the recommended installation location within a virtual space.

[0046] The simulation unit 12 may simulate the user's movement and determine a recommended installation location based on the time the camera is included in the user's field of view. For example, the simulation unit 12 may have the user move through a predetermined movement area (movement trajectory) in a virtual space at a predetermined speed (e.g., a constant speed) and calculate the time the camera is included in the user's field of view. The predetermined movement area is, for example, an area in the real space with a lot of pedestrian traffic. The predetermined movement area may be input by the operator. The simulation unit 12 sets the predetermined movement area based on the operator's input. Alternatively, the simulation unit 12 may recognize an image in the virtual space, estimate an area a predetermined distance from a wall to be an area in the real space with a lot of pedestrian traffic, and set that area as the predetermined movement area.

[0047] The simulation unit 12 calculates a visibility score, which indicates the degree to which the user sees the camera, based on the calculated time. The visibility score corresponds to the length of time the camera is included in the user's field of view. The simulation unit 12 calculates a higher visibility score the longer the time. The simulation unit 12 may also identify a recommended installation location based on the overlap and the visibility score.

[0048] The above is just one example, and the conditions for the recommended installation location can be changed as appropriate. For example, the simulation unit 12 may set a threshold for overlap and identify the recommended installation location based on that threshold. The simulation unit 12 may also set the heights of multiple users as appropriate, depending on the type of store, etc. For example, in facilities for children or some stores, there may be many short users. In such cases, the simulation unit 12 may identify the recommended installation location using the field of view based on short users. In this case, the simulation unit 12 may prioritize the field of view of short users when calculating the overlap of the field of view to determine how easily the camera will be recognized. The simulation unit 12 may also set different thresholds for each height and identify the recommended installation location considering the overlap of the field of view in each height group.

[0049] The display control unit 13 controls the display of information in the information processing device 10 and causes the display unit 14 to display the information. For example, the display control unit 13 displays a display screen that includes a camera view which displays a captured image based on the camera's viewpoint and an object view which displays the field of view based on the object's viewpoint.

[0050] Here, a virtual user is used as an object, so the object view visualizes the field of view based on that user's viewpoint. Therefore, in the following explanation, the object view will be referred to as the "user view." The camera view is a display area that shows the captured image based on the camera's viewpoint. The user view is a display area that shows the field of view based on the user's viewpoint.

[0051] Figure 2 shows an example of a display screen P1 including the camera view CV and the user view UV. The display screen P1 mainly includes the user view UV. The user view UV displays the field of view based on the user U's viewpoint. The user view UV is the area enclosed by a thin solid line frame.

[0052] As shown in the user view UV, camera C1 is mounted on the ceiling. Camera C1 captures the virtual space from the ceiling. Camera C1 captures the area including user U, who is on the near side of the page. As shown in the user view UV, camera C1 is included in user U's field of view.

[0053] Furthermore, the display screen P1 includes a camera view CV superimposed on the user view UV in the upper left portion of the user view UV. The camera view CV displays the captured image based on the viewpoint of camera C1. The camera view CV is the area enclosed by the thick solid line frame. As shown in the camera view CV, the shooting range of camera C1 includes user U.

[0054] The images displayed on the camera view CV and user view UV may be still images or moving images. In this case, the display control unit 13 will display moving images on the camera view CV and user view UV.

[0055] The display control unit 13 may display the camera view CV and user view UV in any display mode. For example, the area ratio of the camera view CV and user view UV is not limited to that shown in Figure 2. The display control unit 13 may display the camera view CV and user view UV in roughly the same area. In addition, the display control unit 13 may display the camera view CV larger than the user view UV depending on the operator's input.

[0056] As shown in Figure 2, the display control unit 13 superimposes the camera view CV and the user view UV on the same display screen P1. This allows the operator to view both the camera view CV and the user view UV simultaneously. In this way, the operator can intuitively understand both how camera C1 looks from the user U's perspective and how camera C1 can capture images of user U.

[0057] The display control unit 13 may determine whether the camera is within the user U's field of view and display accordingly. For example, if the display control unit 13 determines that the camera is included in the user view UV, it may highlight the camera's position in the user view UV. In this case, camera C1 is included in the user view UV. Therefore, the display control unit 13 may highlight the position of camera C1. Similarly, the display control unit 13 may highlight the position of user U in the camera view CV. In this way, the operator can easily grasp the position of camera C1 and the position of user U on the display screen P1.

[0058] The display control unit 13 may display the camera C1 in different ways depending on the user U's field of view. For example, the display control unit 13 may display the position of the camera C1 in different colors depending on whether the field of view containing the camera C1 is the user U's central field of view, effective field of view, or peripheral field of view. For example, the display control unit 13 may display the position of the camera C1 in different colors for each field of view, such as red if the camera C1 is in the user U's central field of view, yellow if it is in the effective field of view, and blue if it is in the peripheral field of view. The display control unit 13 may superimpose colors on the camera C1 on the display screen P1, or it may color-code the camera C1 by surrounding it with a colored frame or the like. This allows the operator to intuitively understand which field of view of the user U the camera C1 is located in.

[0059] Furthermore, the display control unit 13 can display the user U and camera C1 in any display manner that allows for identification, not limited to highlighting or color coding. For example, the display control unit 13 can use methods such as adding text information to camera C1 and user U, making camera C1 and user U blink, or superimposing icons representing camera C1 and user U, respectively. The display control unit 13 may also display a message such as "Camera C1 is within user U's field of view." The display control unit 13 may also display a message such as "Camera C1 is within user U's effective field of view" based on field of view area information acquired from the simulation unit 12.

[0060] In Figure 2, an example is shown where the entire camera C1 is visible from the user U's viewpoint, and the entire user U is visible from the camera C1's viewpoint, but this is not the only example. For example, user U may only see a part of camera C1. Similarly, camera C1 may only be able to capture a part of user U's body. For example, if camera C1 is positioned to capture only the torso of user U, the camera C1 can be positioned in a location that is difficult for user U to see. The display control unit 13 may also superimpose the shooting range of camera C1 onto the user view UV.

[0061] Furthermore, if multiple cameras are installed in the virtual space, the display control unit 13 may display the image captured by the camera that is capturing user U on the display screen. This allows the display control unit 13 to switch to displaying the image captured by another camera that can capture user U if user U moves out of the camera's shooting range.

[0062] Figure 3 shows an example of a display screen P2 including the camera view CV and user view UV at a location different from the location shown in Figure 2. Assume that user U moves from the location shown in Figure 2 to the location shown in Figure 3. When user U moves from the location shown in Figure 2 to the location shown in Figure 3, camera C1, which was capturing user U, can no longer capture user U. If multiple cameras are installed in the virtual space, it is possible that one of the cameras can capture user U.

[0063] Now, suppose the camera tracking user U changes from camera C1 to camera C2. The display control unit 13 stops displaying the image captured by camera C1, which no longer tracks user U, in the camera view CV, and starts displaying the image captured by camera C2, which is now tracking user U. In other words, the display control unit 13 switches the display in the camera view CV from the image captured by camera C1 to the image captured by camera C2.

[0064] For example, the display control unit 13 identifies the camera to be switched to based on the position of the moving user U and the shooting range of each of the multiple cameras. The display control unit 13 may also identify the camera that captures user U by determining whether or not user U has been detected in the captured image of each of the multiple cameras.

[0065] In the example shown in Figure 3, the display control unit 13 displays the camera view CV using the image captured by camera C2, which has just acquired user U. The display control unit 13 also displays the user's field of view, including camera C2, on the user view UV. In this way, the display control unit 13 can appropriately update the camera view CV and user view UV in accordance with the movement of user U. This allows the operator to understand the status of user U's acquisition by the camera in real time.

[0066] The display control unit 13 may highlight the camera view CV or display a message when the camera capturing user U switches. Furthermore, the display control unit 13 may highlight the camera view CV or display a message when the camera no longer captures user U.

[0067] Furthermore, although not shown in the figures, the display control unit 13 may superimpose field of view information indicating the user's field of view onto the user view UV shown in Figures 2 and 3. For example, the display control unit 13 superimposes the central field of view, effective field of view, and peripheral field of view onto the user view UV. The display control unit 13 maps the areas corresponding to the central field of view, effective field of view, and peripheral field of view onto the user view UV.

[0068] The display control unit 13 may display each viewing area in a different manner. For example, the display control unit 13 may display each viewing area in a different color. The display control unit 13 may represent the viewing areas as circular, elliptical, or concentrically extending areas. However, it is not limited to these, and the display control unit 13 may highlight the boundaries of the viewing areas, add text labels to each viewing area, or display each viewing area with a different transmittance. The display control unit 13 may control the display using various methods to make the differences between viewing areas visually distinguishable.

[0069] In this way, the operator can easily understand the correspondence between the user view UV and the user U's field of view. For example, if there is a desire that "the camera does not need to be positioned within the user's central field of view, but should be positioned within their peripheral field of view," the operator can consider the camera's placement by referring to the field of view information.

[0070] Furthermore, the display control unit 13 may change the field of view information in accordance with any change in the user U's viewpoint. Examples of changes in the user U's viewpoint include when the user U moves, when they move from a seated position to a standing position, when they move from a standing position to a seated position, or when the user U's height setting is changed.

[0071] For example, if user U moves, the position of user U's eyes changes, and therefore the position of the viewpoint changes. Similarly, if user U moves from a seated position to a standing position, or from a standing position to a seated position, the eye height changes, and therefore the position of the viewpoint changes. Furthermore, if the height setting of user U used in the simulation is changed, the eye height changes, and therefore the position of the viewpoint changes. The display control unit 13 updates the field of view area information mapped to the user view UV shown in Figures 2 and 3 in accordance with the changes in the position of user U's viewpoint.

[0072] Furthermore, the display control unit 13 may also display a free-viewpoint view FV (not shown) based on a free viewpoint different from the user U's viewpoint on display screens P1 and P2. The free-viewpoint view FV is a viewpoint from which the operator can check the situation in the virtual space from any position or angle. The display control unit 13 displays the free-viewpoint view FV as a viewpoint independent of the camera view CV and the user view UV. By using the free-viewpoint view FV, the operator can check the camera placement and the user's field of view in the virtual space from a third-person viewpoint.

[0073] The display control unit 13 may display the user view UV and the free-viewpoint view FV on the same screen. For example, in the example shown in Figure 2, the display control unit 13 may place the user view UV on the left half of the display screen P1 and the free-viewpoint view FV on the right half of the display screen P1. This allows the display control unit 13 to display the camera view CV, user view UV, and free-viewpoint view FV on the display screen P1. This allows the operator to simultaneously view the images of the camera view CV, user view UV, and free-viewpoint view FV. The display control unit 13 may also display the user view UV and free-viewpoint view FV side by side, one above the other.

[0074] The display control unit 13 may also enable switching between the user view UV and the free-viewpoint view FV. For example, the display control unit 13 switches the display on the display screen P1 from the user view UV to the free-viewpoint view FV, or from the free-viewpoint view FV to the user view UV, in response to a switching request from the operator. This allows the operator to check the camera's installation status and the user's field of view from different viewpoints as needed.

[0075] Although this example uses the camera view CV to be displayed at all times, it is not limited to this. The display control unit 13 may adjust the display mode of the display screen P1 to not display the camera view CV at the operator's request. The display control unit 13 may also display the free-viewpoint view FV on the display screen P2 in the same manner as the display screen P1. Furthermore, the display control unit 13 may superimpose the camera's shooting range and the user's field of view on the free-viewpoint view FV.

[0076] Returning to Figure 1, the explanation continues. The display unit 14 displays an image that can be output by the processing according to this disclosure. The display unit 14 is, for example, a display. The display unit 14 may also be a touch panel equipped with the functions of an input unit.

[0077] For example, the display unit 14 displays the camera view and the user view. The display unit 14 may also display a free-viewpoint view. In addition, the display unit 14 may display the recommended installation position proposed by the simulation unit 12. In this case, the display unit 14 may superimpose the recommended installation position onto the camera view or the user view. In this way, the operator can easily grasp the recommended installation position.

[0078] Furthermore, the display unit 14 may display the aforementioned overlap and field of view information in correspondence with the recommended installation location. In this way, the operator can visually understand the reasons why the recommended installation location is recommended.

[0079] The input unit 15 receives input of information used in the processing related to this disclosure. Specifically, the input unit 15 receives input from the operator. The input unit 15 is an input device capable of operating a camera and a user in a virtual space. The input unit 15 receives operations from the operator and acquires operation information corresponding to those operations. The input unit 15 outputs the operation information to the simulation unit 12.

[0080] The input unit 15 may be, for example, a mouse, keyboard, touch panel, smartphone, tablet, game controller, voice input device, gesture input device, VR (Virtual Reality) controller, or eye-tracking input device. An example of operating the input unit 15 is shown below. Note that in (1) and (2), a mouse is used as the input unit 15, and in (3), a keyboard is used as the input unit 15.

[0081] <An example of operation in the input unit 15> (1) Operations when the mouse is hovering over the camera view Left click: Switch cameras in camera view. Wheel: Zooms in on the camera view image.

[0082] (2) Operations when the mouse is hovering over the user view Left-drag: Moves the user's viewpoint. Wheel: Moves the user forward and backward.

[0083] (3) Keyboard operation "T" key: Move camera "R" key: Camera pan and tilt operation "E" key: Toggles the display of the guide. "M" key: Switches between user view and free-view view.

[0084] In this way, by using one mouse or one keyboard as the input unit 15, the input unit 15 can accept operations for both the camera view and the user view. The inputs to the input unit 15 and the operations corresponding to those inputs can be changed as desired. For example, in the above example, by assigning operations to adjacent keys on an English keyboard, such as "T", "R", and "E", the operator can operate intuitively, but the assignment of each key can be changed according to the operator's convenience.

[0085] Furthermore, when a terminal device such as a smartphone is used as the input unit 15, the input unit 15 may accept operations such as pinching and swiping and acquire operation information corresponding to these operations. For example, the input unit 15 may acquire operation information corresponding to zooming in or zooming out by a pinch operation. The input unit 15 may also acquire operation information corresponding to changing the viewpoint by a swipe operation. The input unit 15 may also acquire operation information corresponding to switching cameras by a tap operation, and so on.

[0086] The input unit 15 outputs this operation information to the simulation unit 12 and the display control unit 13. This allows the operator to change the viewpoint and control the camera within the virtual space via the input unit 15.

[0087] (Processing by the information processing device 10) The process performed by the information processing device 10 will be explained with reference to Figure 4. Figure 4 is a flowchart showing the flow of the process performed by the information processing device 10.

[0088] First, the virtual space generation unit 11 generates a virtual space (S11). Although omitted in Figure 4, after the virtual space is generated, the simulation unit 12 constantly receives input from the input unit 15 and performs a simulation within the virtual space based on the input. For example, the simulation unit 12 acquires operation information corresponding to keyboard operations received by the input unit 15 and performs a simulation based on that operation information. For example, the simulation unit 12 performs simulations that include changing the camera's position within the virtual space and the user's movement.

[0089] Next, the display control unit 13 displays the camera view on the display unit 14 (S12). The display control unit 13 also displays the user view on the same screen as the camera view (S13). The order of steps S12 and S13 may be reversed. In addition to the camera view and user view, the display control unit 13 may also display a free viewpoint view on the same screen. Alternatively, the display control unit 13 may display two or more views from the camera view, user view, and free viewpoint view in a switchable manner. In this case, the display control unit 13 may display a switch button on the display screen to receive a switch request from the operator.

[0090] Next, the display control unit 13 determines whether or not there has been a request to switch the display screen (S14). The display control unit 13 receives the switch request from the operator via the input unit 15. A request to switch the display screen may, for example, request a switch between the user view and the free-viewpoint view. Alternatively, a request to switch the display screen may request a switch between a state in which the field of view area information is not superimposed on the user view and a state in which it is superimposed.

[0091] If it is determined that there is no request to switch display screens (NO in S14), the process proceeds to step S16. If it is determined that there is a request to switch display screens (YES in S14), the display control unit 13 displays the corresponding display screen on the display unit 14 (S15). For example, the display control unit 13 switches between the user view and the free-viewpoint view. The display control unit 13 also switches the display on and off of the superimposed display of field-of-view area information in the user view.

[0092] Next, the simulation unit 12 determines whether or not to terminate the simulation (S16). For example, the simulation unit 12 receives input from the operator via the input unit 15 indicating that the simulation should be terminated. If it determines that the simulation should be terminated (YES in S16), the process ends. If it determines that the simulation should not be terminated (NO in S16), the process returns to step S14 and repeats the subsequent processing.

[0093] Although not shown in the diagram, the simulation unit 12 may identify recommended installation locations and propose these locations to the operator. For example, the simulation unit 12 sets up multiple users with different viewpoints and uses the field of view areas of these multiple users to calculate the degree of overlap, indicating how much of the predetermined installation location is included in the field of view of multiple users. The simulation unit 12 identifies locations with a relatively high or relatively low degree of overlap in the field of view and identifies recommended installation locations based on the identification results. The simulation unit 12 then proposes these recommended installation locations to the operator.

[0094] The simulation unit 12 may output the proposed content in any manner. For example, the simulation unit 12 may output the proposed content by displaying it on the display unit 14. The simulation unit 12 may also transmit the proposed content to a terminal device used by the operator (e.g., a smartphone) via a network (not shown). The network includes, for example, an internet connection or a wireless communication network, but the type of communication is not limited to these. The simulation unit 12 may also output the proposed content as audio via an audio output unit (not shown).

[0095] As described above, the information processing device 10 according to this disclosure can display a display screen that includes a camera view that displays captured images based on the camera's viewpoint and an object view that displays the field of view based on the object's viewpoint. This allows the operator to easily check images captured at the location where they want to install the camera in real space, without actually installing the camera in real space. With this configuration, the information processing device 10 can appropriately simulate the camera's installation position, taking into account how the camera appears from the viewpoint of a user utilizing the space.

[0096] Each functional component of the information processing device 10 described above may be implemented by hardware (e.g., hardwired electronic circuits) or by a combination of hardware and software (e.g., a combination of an electronic circuit and a program that controls it). For example, the present disclosure can also be implemented by having a CPU execute a computer program.

[0097] The program, when loaded into a computer, includes a set of instructions (or software code) for causing the computer to perform one or more of the functions described in the embodiments. The program may be stored in various types of non-transitory computer-readable medium or tangible storage medium. Examples, but not limited to, include RAM (Random-Access Memory), ROM (Read-Only Memory), flash memory, SSD (Solid-State Drive), or other memory technologies, CD-ROM, DVD (Digital Versatile Disc), Blu-ray® disc, or other optical disc storage, magnetic cassette, magnetic tape, magnetic disk storage, or other magnetic storage devices. The program may also be transmitted over various types of transient computer-readable medium or communication medium. Examples, but not limited to, include transient computer-readable medium or communication medium, including electrically, optically, acoustically, or otherwise propagating signals.

[0098] This disclosure is not limited to the embodiments described above, and can be modified as appropriate without departing from the spirit of the disclosure. For example, Figure 1 illustrates an example in which the information processing device 10 includes a virtual space generation unit 11, but is not limited to this. The information processing device 10 may perform the processing related to this disclosure using a virtual space generated by another device.

[0099] Furthermore, although Figure 1 shows the display unit 14 and input unit 15 inside the information processing device 10, this is not the only way to do so. The display unit 14 and input unit 15 may be provided separately from the information processing device 10. [Explanation of symbols]

[0100] 10 Information Processing Devices 11. Virtual Space Generation Unit 12 Simulation Department 13 Display Control Unit 14 Display section 15 Input section C1, C2 Camera CV Camera View FV (Free View) P1, P2 display screen U User UV User View

Claims

1. A simulation unit that simulates the camera's installation and shooting conditions using a virtual space that includes a camera that takes pictures within the virtual space and objects that can move within the virtual space. The system includes a display control unit that displays a display screen including a camera view that displays a captured image based on the viewpoint of the camera, and an object view that displays a field of view based on the viewpoint of the object. Information processing device.

2. If multiple cameras are installed in the virtual space, the display control unit will display the images captured by the cameras that are capturing the objects on the display screen. The information processing apparatus according to claim 1.

3. The simulation unit classifies the field of view of the object into multiple field of view regions based on the object's viewpoint. The display control unit overlays the field of view information indicating the field of view onto the object view, and changes the field of view information in accordance with the change when the viewpoint position of the object changes. The information processing apparatus according to claim 1 or 2.

4. The display control unit further displays a free viewpoint view based on a free viewpoint different from the viewpoint of the object, and displays the object view and the free viewpoint view on the same screen, or displays the object view and the free viewpoint view in a switchable manner. The information processing apparatus according to claim 1 or 2.

5. A simulation step in which the camera's installation and shooting conditions are simulated using a virtual space that includes a camera for taking pictures within the virtual space and objects that can move within the virtual space, The computer is instructed to perform a display control step that causes a display screen to be displayed, which includes a camera view that displays a captured image based on the viewpoint of the camera, and an object view that displays a field of view based on the viewpoint of the object. program.

Images