Information processing device and information processing method

Abstract

Displaying images from multiple devices in a space-saving manner. [Solution] The information processing device is a device that is communicatively connected to source devices installed in each partitioned area of ​​a predetermined space. The information processing device comprises a display unit having a first screen, a display control unit that displays an image on the first screen, and an acquisition unit that acquires a shared source image which is a display image on a second screen of a source device. When the display control unit detects a first selection operation to select a partitioned area, it displays the shared source image displayed on the second screen of each source device installed in the selected partitioned area on the first screen.

Description

Technical Field

[0001] This disclosure relates to an information processing apparatus and an information processing method.

Background Art

[0002] For example, Patent Document 1 describes a system that can check the playback status of content in a plurality of in-vehicle devices installed in a vehicle interior. In the system described in Patent Document 1, for example, the playback status of content in all in-vehicle devices in the vehicle interior is displayed on the screen of one in-vehicle device that serves as a master.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in Patent Document 1, for example, the space within the screen required to display the playback status of content increases in proportion to the number of in-vehicle devices installed in the vehicle interior. The in-vehicle devices described in Patent Document 1 have room for improvement in terms of securing a display space for information other than the playback status of content in a plurality of devices.

[0005] In view of the above circumstances, an embodiment of this disclosure aims to provide an information processing apparatus and an information processing method capable of displaying the display images of a plurality of devices in a space-saving manner.

Means for Solving the Problems

[0006] An information processing device according to one embodiment of the present disclosure is a device that is communicatively connected to source devices installed in each partitioned area of ​​a predetermined space. The information processing device comprises a display unit having a first screen, a display control unit that displays an image on the first screen, and an acquisition unit that acquires a shared source image which is a display image on a second screen of a source device. When the display control unit detects a first selection operation to select a partitioned area, it displays the shared source image displayed on the second screen of each source device installed in the selected partitioned area on the first screen. [Effects of the Invention]

[0007] According to one embodiment of the present disclosure, an information processing device and an information processing method are provided that can display images of multiple devices in a space-saving manner. [Brief explanation of the drawing]

[0008] [Figure 1] This figure schematically shows a vehicle equipped with an in-vehicle system according to one embodiment of the present disclosure. [Figure 2] This is a block diagram showing the configuration of an information processing device according to one embodiment of the present disclosure. [Figure 3] This figure shows an example of a display screen displayed on the screen of the display unit of an information processing device according to one embodiment of the present disclosure. [Figure 4] This figure shows an example of a display screen displayed on the screen of the display unit of an information processing device according to one embodiment of the present disclosure. [Figure 5] This figure shows an example of a display screen displayed on the screen of the display unit of an information processing device according to one embodiment of the present disclosure. [Figure 6] This figure shows an example of a display screen displayed on the screen of the display unit of an information processing device according to one embodiment of the present disclosure. [Figure 7] This figure shows an example of a display screen displayed on the screen of the display unit of an information processing device according to one embodiment of the present disclosure. [Figure 8] This flowchart shows the processing performed by the control unit of the information processing device in one embodiment of the present disclosure. [Figure 9]This flowchart shows the processing performed by the control unit of the information processing device in one embodiment of the present disclosure. [Figure 10] This figure shows an example of a display screen displayed on the screen of the display unit of an information processing device according to one embodiment of the present disclosure. [Figure 11] This figure shows an example of a display screen that is displayed on the screen of the display unit of the information processing device in Modification 1 of this disclosure. [Figure 12] This figure shows an example of a display screen that is displayed on the screen of the display unit of the information processing device in Modification 2 of this disclosure. [Modes for carrying out the invention]

[0009] The following description relates to an information processing apparatus and an information processing method according to one embodiment of the present disclosure. Common or corresponding elements are denoted by the same or similar reference numerals, and redundant descriptions are omitted or simplified as appropriate.

[0010] Figure 1 is a schematic diagram showing a vehicle V1 on which an in-vehicle system 1 according to one embodiment of the present disclosure is installed. Vehicle V1 is an 8-seater passenger car and is an example of a mobile vehicle. Three rows of seats are installed in the vehicle interior. The first row of seats consists of the driver's seat and the passenger seat. The second and third rows of seats are bench seats that can accommodate three people.

[0011] The seating layout inside the vehicle is not limited to that shown in Figure 1. For example, four or more rows of seats may be installed inside the vehicle. Bench seats may accommodate four or more people. Vehicle V1 is not limited to a regular passenger car; it may also be a large vehicle such as a bus. Vehicle V1 may be replaced with another mode of transport, such as an airplane.

[0012] The in-vehicle system 1 includes a plurality of information processing devices 10. The information processing device 10 is an in-vehicle device capable of displaying various objects and is an example of a computer. For example, eight information processing devices 10 are installed in the vehicle interior. The information processing device 10 installed for the left seat in the first row (for example, the driver's seat) is denoted as "information processing device 10A". The information processing device 10 installed for the right seat in the first row (for example, the passenger seat) is denoted as "information processing device 10B". The three information processing devices 10 installed for the bench seat in the second row are denoted as "information processing device 10C", "information processing device 10D", and "information processing device 10E" in order from the left seat side. The three information processing devices 10 installed for the bench seat in the third row are denoted as "information processing device 10F", "information processing device 10G", and "information processing device 10H" in order from the left seat side.

[0013] The number and installation positions of the information processing devices 10 are not limited to those illustrated in FIG. 1. For example, the information processing device 10 may not be installed for some seats, or two or more information processing devices may be installed for some seats.

[0014] The information processing device 10 is not limited to an in-vehicle device. The information processing device 10 may be a portable device such as a smartphone, a tablet terminal, or a notebook PC (Personal Computer). For example, the information processing device 10A may be an in-vehicle device, and the other information processing devices 10B to 10H may be smartphones or tablet terminals. That is, the information processing devices 10A to 10H may be devices in different forms.

[0015] FIG. 2 is a block diagram showing the configuration of the information processing device 10 according to an embodiment of the present disclosure. As shown in FIG. 2, the information processing device 10 includes a control unit 100, a memory 110, an HMI (Human Machine Interface) 120, a communication interface 130, and a display unit 140. The information processing device 10 may include other configurations not shown in FIG. 2. That is, there is flexibility in the form of the information processing device 10, and various design changes are possible.

[0016] The control unit 100 is configured as, for example, an LSI (Large Scale Integration), and includes a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), and a DSP (Digital Signal Processor). The control unit 100 may be referred to as, for example, a processor, an MPU (Micro Processor Unit), an MCU (Micro Controller Unit), etc.

[0017] The control unit 100 reads various programs and data stored in the built-in ROM or the memory 110, and uses the RAM as a work area to control the information processing apparatus 10.

[0018] The memory 110 is, for example, a non-volatile semiconductor memory such as an auxiliary storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), a flash memory, an EPROM (Erasable Programmable ROM), or an EEPROM (Electrically Erasable Programmable ROM).

[0019] For example, an application 110A, which is an example of a program, is stored in the memory 110. By executing the application 110A by the control unit 100, various processes (for example, various image processes) according to an embodiment of the present disclosure are executed. Each of the information processing apparatuses 10A to 10H can operate as a source device that is a mirroring source (or, in other words, a sharing source of an image) by executing the application 110A, and can also operate as a sink device that is a mirroring destination (or, in other words, a sharing destination of an image).

[0020] Application 110B could be, for example, an application for streaming video playback or an application for displaying a navigation screen. Figure 2 shows two applications, 110A and 100B, but the information processing device 10 may store more applications. Different applications may be stored in the memories 110 of each of the information processing devices 10A to 10H.

[0021] HMI120 (Human Machine Interface) can encompass various user interfaces, including hardware, software, or combinations thereof. Examples include mechanical switches (such as mechanical, capacitive non-contact, and membrane types), buttons, knobs, wheels, and other mechanical controls; GUIs (Graphical User Interfaces) provided in touch panel environments; and remote controllers. HMI120 may also include interfaces for gesture recognition, voice recognition, and other similar functions.

[0022] The communication interface 130 is an interface that enables the information processing device 10 to communicate with various external devices using wireless communication standards such as Wi-Fi, Wi-Fi Direct (registered trademark), and Bluetooth (registered trademark). The information processing devices 10A to 10H are connected to each other via the communication interface 130 and are also connected to servers, clouds, etc., located on external networks, enabling them to communicate with each other.

[0023] Some of the various processes according to one embodiment of this disclosure may be performed by an external device connected to the information processing device 10 via a communication interface 130, for example, instead of the control unit 100.

[0024] The display unit 140 includes a screen 142 composed of, for example, an LCD (Liquid Crystal Display) panel, an organic EL (Electro Luminescence) panel, or an LED (Light Emitting Diode) panel. The screen 142 is equipped with a touch panel.

[0025] The user can perform various touch operations on the display, such as touch-on (touching the screen 142 with a finger), touch-off (lifting the finger from the screen 142), flick (swiping the finger across the screen 142), swipe (sliding), drag, and drop. The control unit 100 detects the coordinates on the screen 142 that are being touched and executes the processing associated with the detected coordinates.

[0026] When describing a component of a specific information processing device 10 from among the information processing devices 10A to 10H, the corresponding alphabet is appended to the end. For example, the display unit 140 and screen 142 of information processing device 10A are written as "display unit 140A" and "screen 142A," respectively.

[0027] The upper diagrams in Figures 3 to 5 show examples of display screens shown on screen 142A of the information processing device 10A. The hands shown in each diagram showing the example display screens represent the hands of the user performing touch operations. The arrows shown in hatching indicate the movements of the user's hands performing touch operations. The lower diagrams in Figures 3 to 5 are simplified diagrams showing the installation positions of the information processing devices 10A to 10H (screens 142A to 142H) within the vehicle interior. In the lower diagrams, the differences in the display content of screen 142 of each information processing device 10 are represented by differences in the color and hatching of each block.

[0028] In the examples described below, information processing device 10A operates as a sink device, and information processing devices 10B to 10H operate as source devices. Specifically, the display unit 140A of information processing device 10A is an example of a display unit having a first screen. The control unit 100A of information processing device 10A is an example of a display control unit that displays an image on the first screen. The user can appropriately configure which information processing device 10 operates as a sink device or a source device.

[0029] Any reference to elements using designations such as “First,” “Second,” etc., as used in this disclosure, does not generally limit the quantity or order of those elements. These designations are used for convenience to distinguish between two or more elements. Therefore, references to the First and Second elements do not, for example, imply that only two elements are adopted, or that the First element must precede the Second element.

[0030] In the example shown in Figure 3, objects C1 and C2 are displayed on screen 142A. Objects display information about what the application has done. Objects may also be symbols representing such information, or indicators of the type of application. The shape of the objects displayed on screen 142 is, for example, a rectangle. The user can change the display size of an object by, for example, touching and swiping the corners of the object.

[0031] Objects may display, for example, video content, navigation content that displays information related to navigation functions, parking assistance content that displays an overhead view of the vehicle when parked, audio player content for operating audio functions, radio operation content for tuning radio stations, in-car control content for adjusting the temperature and airflow of the air conditioner installed in the vehicle, and vehicle-related content for displaying and setting various information about the vehicle (e.g., speedometer, tachometer).

[0032] For example, let's consider a scenario where a parent is seated in the driver's seat and children (child users) are seated in other seats. We will explain an example of screen operation in the in-vehicle system 1. For convenience, the parent seated in the driver's seat will be referred to as "Parent User A".

[0033] For example, parent user A and child user can individually enjoy content on information processing devices 10 installed in their respective seats. Parent user A may want to know what content the child user is watching. However, it is difficult for parent user A, who is seated in the driver's seat, to directly see the screen 142 of the information processing device 10 in the other seat. Therefore, in this embodiment, parent user A can see the display content of the screen 142 of the information processing device 10 in the other seat through the screen 142A of the information processing device 10A.

[0034] Furthermore, it is difficult for a child user unfamiliar with operating the information processing device 10 to perform operations such as selecting and playing new content or changing the layout of objects displayed on screen 142. Parent user A cannot physically reach the screen 142 of the child user's information processing device 10. Therefore, it is also difficult for parent user A to directly touch and operate the screen 142 of the child user's information processing device 10. Thus, in this embodiment, parent user A is configured to remotely control the child user's information processing device 10 via the information processing device 10A.

[0035] A first selection operation, which is a trigger for confirming the displayed content and performing remote control, is illustrated in Figure 3. In the example in Figure 3, parent user A touches the right-leaning area 200R near the bottom edge of the screen with their finger and swipes vertically upward by a predetermined distance or more. As a result, the group window GW1 appears, for example, as if pulled vertically upward from the bottom edge of the screen. In this embodiment, the frame of the group window GW1 is not visible on the screen. In another embodiment, the frame of the group window GW1 may be visible on the screen.

[0036] The first selection operation is the operation of selecting a partitioned area separated from a predetermined space. Areas R1 to R4 shown in Figure 1 are examples of partitioned areas separated from the vehicle interior, which is an example of a predetermined space. Area R1 includes the passenger seat. Area R2 includes the right row of rear seats (second row right seat and third row right seat). Area R3 includes the center row of rear seats (second row center seat and third row center seat). Area R4 includes the left row of rear seats (second row left seat and third row left seat).

[0037] The first selection operation illustrated in Figure 3 is the operation to select region R2. The first selection operation is not limited to this operation (i.e., the operation of swiping a predetermined distance vertically upwards within the right-side region 200R). The first selection operation may also be the operation to select region R1 (i.e., the operation of swiping a predetermined distance horizontally to the left within the right-side region 200F near the right edge of the screen). The first selection operation may also be the operation to select region R3 (i.e., the operation of swiping a predetermined distance vertically upwards within the central region 200C near the bottom edge of the screen). The first selection operation may also be the operation to select region R4 (i.e., the operation of swiping a predetermined distance vertically upwards within the left-side region 200L near the bottom edge of the screen).

[0038] In Figures 3 to 6, for convenience, the right-aligned region 200R, the central region 200C, the left-aligned region 200L, and the right-edge region 200F are illustrated, but these regions are not elements that are displayed on the screen. In another embodiment, these regions may be displayed on the screen.

[0039] Group window GW1 is associated with the right rear seat (in other words, area R2). Group window GW1 displays components 210E and 210H, which correspond to the information processing devices 10E and 10H installed in the right rear seat. Each component within the group window is collectively referred to as "component 210". Component 210E displays column information (for example, "2" indicating the second column) and a small window 220E. Component 210H displays column information (for example, "3" indicating the third column) and a small window 220H. The small windows within component 210 are collectively referred to as "small windows 220".

[0040] The small window 220 displays the display screen of the corresponding information processing device 10, for example, using MIRACAST®. Specifically, the small window 220E displays the display screen of the information processing device 10E (i.e., a mirrored image of the source screen 142E). The small window 220H displays the display screen of the information processing device 10H (i.e., a mirrored image of the source screen 142H). In another embodiment, the display screen (mirrored image) of the source information processing device 10 may be displayed in the small window 220 (and window W1 described later) using another method such as Android® Auto or CarPlay®.

[0041] A similar group window will appear if you perform the first selection operation on other areas. For example, if you perform the first selection operation on the central area 200C, the group window GW2 associated with the center row of the rear seats (in other words, area R3) will be displayed (see Figure 5). If you perform the first selection operation on the left-side area 200L, the group window associated with the left row of the rear seats (in other words, area R4) will be displayed. If you perform the first selection operation on the right-side area 200F, the group window associated with the passenger seat (in other words, area R1) will be displayed.

[0042] For example, the group window GW2 displays components 210D and 210G corresponding to the information processing devices 10D and 10G installed in the center row of the rear seats. Component 210D displays column information and a small window 220D. Component 210G displays column information and a small window 220G. The small window 220D displays a mirrored image shared by the information processing device 10D. The small window 220G displays a mirrored image shared by the information processing device 10G.

[0043] Parent user A can determine what content the child user seated in the selected area (right rear seat in the example in Figure 3, and center rear seat in the example in Figure 5) is viewing by viewing the small window 220. In other words, parent user A can determine what content the child user is viewing without directly viewing the screen 142 of the information processing device 10 in the other seats.

[0044] Parent user A can limit the area within the vehicle interior and display a mirrored image of the information processing device 10 belonging to that limited area on screen 142A. Therefore, even when there are many information processing devices 10 installed in the vehicle interior, the space on screen 142 required to display the content playback status can be kept small (i.e., limited to the group window). Parent user A can, for example, keep track of what content the child user is viewing while ensuring display space for objects C1 and C2.

[0045] For example, if parent user A has performed the first selection operation and then moves their finger over component 210 in the group window, component 210 at the destination of the finger will become selected. In the example in Figure 4, component 210E will become selected.

[0046] Parent user A can drag and drop the small window 220 of the selected component 210. In the examples in Figures 4 and 5, parent user A drags the small window 220E into the central area 200C. Parent user A performs the first selection operation while dragging the small window 220E. As a result, the group window GW2 is displayed on screen 142A.

[0047] In the example shown in Figure 5, parent user A drops a small window 220E onto component 210D in group window GW2. When information processing device 10A detects this drop operation, it instructs information processing devices 10D and 10E to share the mirrored image displayed in the small window 220E with the information processing device 10D corresponding to component 210D. As a result, information processing devices 10D and 10E communicate with each other, and the display screen of information processing device 10E is displayed as a mirrored image on the screen 142D of information processing device 10D.

[0048] In the example shown in the upper part of Figure 6, parent user A drags and drops small window 220E outside the group window GW1. As a result, small window 220E is resized to a predetermined size and placed at the drop location. As shown in the lower part of Figure 6, the resized small window 220E is labeled "Window W1". Window W1 then displays the display screen of the information processing device 10E, for example, using MIRACAST. For example, window W1 displays a mirrored image in which objects C3 and C4 are arranged in the same layout as screen 142E.

[0049] Window W1 is displayed at a size that occupies, for example, one-quarter of the entire screen 142A. For example, if screens 142A and 142E have the same resolution, the mirrored image from the source information processing device 10E is reduced to 50% of its original size and displayed within window W1. For example, if screen 142A is 4K (3,840 x 2,160) and screen 142E is FHD (1,920 x 1,080), the mirrored image from the source information processing device 10E is displayed within window W1 at its original size. In other words, the mirrored image is resized based on the resolutions of the source and destination screens before being displayed within the window.

[0050] Parent user A can change the display size of window W1 to any size by, for example, touching a corner of window W1 with their finger and sliding it in any direction.

[0051] The information processing device 10 supports, for example, UIBC (User Input Back Channel). The information processing device 10A, which operates as a sink device, sends an event corresponding to the parent user A's operation on the mirrored image displayed in the window to the information processing device 10, which operates as a source device. The information processing device 10, which operates as a source device, executes control in response to the event.

[0052] For example, when parent user A touches the play button on object C3 displayed in window W1, a corresponding control command is sent from information processing device 10A to information processing device 10E. The information processing device 10E then starts playing the content of object C3 (for example, video content) according to the received control command. In other words, window W1 also serves as a GUI that accepts operations from parent user A on the screen 142E of the information processing device 10E, which is located remotely.

[0053] Parent user A can change the layout of objects displayed in window W1. In the example shown in the upper part of Figure 7, parent user A touches object C2 displayed on screen 142A with their finger for a certain period of time (i.e., long press). This creates a draggable object C2a, which is displayed on screen 142A. Object C2a is, for example, a reduced-size copy of object C2 (e.g., 50% of the size of object C2) and is displayed in front of object C2.

[0054] In the example shown in the upper part of Figure 7, parent user A drags object C2a displayed on screen 142A and drops it into window W1. This places object C2a at the drop location. When object C2a is placed at the drop location, it is resized, for example, to fit into an empty space within window W1.

[0055] The information processing device 10A, acting as a sink device, instructs the information processing device 10E to display window W2 according to the drag or drop position of object C2a within window W1, and also generates a mirrored image of object C2a and transfers it to the information processing device 10E. As a result, as shown in the lower part of Figure 7, objects C3 and C4 and window W2 (more specifically, the windowed object C2a) are displayed on the screen 142E of the information processing device 10E. In other words, some of the display content of the sink device is transferred to the source device and displayed as a mirrored image.

[0056] Thus, according to this embodiment, an object (for example, an object C2a displayed on screen 142A) can be added from the sink device side (for example, the information processing device 10A), which is the mirroring destination, to the screen of the source device (for example, the screen 142E of the information processing device 10E). Since the layout of the objects displayed on the source device can be changed on the sink device side, the usability of the sink device side is improved.

[0057] Parent user A can, for example, change the layout of objects displayed on screen 142E by long-pressing any object in window W1 and dragging and dropping it to any position within window W1. Parent user A can also delete objects displayed on screen 142E by long-pressing any object in window W1 and performing a predetermined delete operation.

[0058] In this way, parent user A can easily perform operations on the screen 142 of the information processing device 10, which is physically out of reach, via the information processing device 10A.

[0059] Figure 8 is a flowchart showing the process executed by the control unit 100A of the information processing device 10A, which is a sink device, in one embodiment of the present disclosure. For example, when the power of the information processing device 10A is turned on, the execution of the process shown in Figure 8 begins.

[0060] The steps in the flowchart shown in this embodiment may be rearranged, provided they are consistent with each other. Furthermore, the steps in the flowchart shown in this embodiment may be executed in parallel or concurrently, provided they are consistent with each other. For example, while this disclosure presents various steps in an exemplary order, it is not limited to this order.

[0061] The control unit 100A performs a detection process (step S101). Specifically, the control unit 100A detects occupants in the interior of a vehicle V1, which is an example of a moving object (i.e., the interior of a vehicle, which is an example of a predetermined space). The control unit 100A detects the presence or absence of occupants in each seat, for example, using a Driver Monitoring System (DMS) and pressure sensors installed in each seat. The control unit 100A considers the information processing device 10 corresponding to the seat in which an occupant has been detected as the information processing device 10 currently in use.

[0062] Thus, the control unit 100A operates as a detection unit that detects the information processing device 10 currently in use from among a plurality of information processing devices 10 (an example of a source device) installed in a predetermined space.

[0063] The control unit 100A performs partitioning (step S102). Specifically, the control unit 100A partitions areas from inside the vehicle based on the installation location of the information processing device 10 in use, which was detected in the detection process (step S101). For example, if the information processing device 10 in use is detected in all seats, the control unit 100A partitions four areas R1 to R4 from inside the vehicle, as illustrated in Figure 1. For example, if the information processing device 10 in use is detected only in the second-row right seat, the second-row left seat, and the third-row right seat, the control unit 100A partitions two areas from inside the vehicle (an area including the second-row right seat and the third-row right seat, and an area including the second-row left seat).

[0064] In this way, the control unit 100A operates as a partitioning unit that partitions a partitioned area (for example, areas R1 to R4) from the vehicle interior (an example of a predetermined space) based on the installation position of the information processing device 10 (an example of a source device) that is in use, as detected in the detection process (step S101).

[0065] The control unit 100A waits for the first selection operation (step S103). When the control unit 100A detects the first selection operation (step S103: YES), it acquires a mirrored image (step S104). In the example in Figure 3, the control unit 100A requests the display screen, i.e., the mirrored image, from the information processing devices 10E and 10H, which are associated with the first selection operation to select an area including the right rear seat row (an example of a partitioned area, for example, area R2). Upon receiving the request, the information processing devices 10E and 10H transfer the mirrored image to the information processing device 10A. As a result, the control unit 100A, which is operating as an acquisition unit, acquires the mirrored image (an example of a shared source image, which is the display image of the second screen) of the information processing devices 10E and 10H (an example of a source device).

[0066] The control unit 100A displays a group window (step S105). In the example shown in Figure 3, the control unit 100A displays a group window GW1 in which a mirrored image of the information processing device 10E is embedded in the small window 220E of component 210E, and a mirrored image of the information processing device 10H is embedded in the small window 220H of component 210H.

[0067] Thus, the control unit 100A operates as a display control unit. When the control unit 100A, operating as a display control unit, detects the first selection operation, it displays the shared source image (mirrored image) displayed on the screen 142 (an example of the second screen) of each information processing device 10 (an example of a source device) installed in the selected partition area (e.g., area R2) on the screen 142A (an example of the first screen). In addition, the control unit 100A, operating as a display control unit, displays the mirrored image of each information processing device 10 installed in the partition area selected in the first selection operation at a first magnification so that it fits in the small window 220.

[0068] Figure 9 is a flowchart showing the process executed by the control unit 100A of the information processing device 10A, which is a sink device, in one embodiment of the present disclosure. For example, when a second selection operation is detected, the execution of the process shown in Figure 9 is started.

[0069] The second selection operation (a predetermined first operation) is to drag and drop any one small window 220 displayed in the group window to any position outside the group window. In the example in Figure 6, the second selection operation is to drag and drop one small window 220E (in other words, the mirrored image of the information processing device 10E) from among the small windows 220E and 220H (in other words, the mirrored image of the information processing device 10H) displayed in the group window GW1. That is, the second selection operation is to select one shared source image from among the shared source images displayed on the second screen of each source device installed in the partition area selected in the first selection operation.

[0070] When the control unit 100A detects a second selection operation (in other words, a predetermined first operation), it displays a window (an example of a first window, for example, window W1) on the screen 142A and displays the mirrored image selected in the second selection operation in this window (steps S201 to S202).

[0071] Specifically, the control unit 100A, which operates as a display control unit, displays the mirrored image selected in the second selection operation (i.e., the mirrored image in the small window 220 displayed at the first magnification) in the first window (for example, in window W1) at a second magnification greater than the first magnification. In addition, the control unit 100A, which operates as a display control unit, adjusts the display size of the mirrored image (an example of a shared source image) based on the resolution of screen 142A (an example of a first screen) and screen 142 of the information processing device 10 that shares the information (for example, to adjust it to a size that occupies one-quarter of the entire screen 142A), and displays the resized mirrored image in the first window (for example, window W1).

[0072] The control unit 100A waits for a predetermined second operation (step S203). The second operation is an operation on a first object (e.g., object C2a) displayed on screen 142A (an example of the first screen). More specifically, the second operation is to drag the first object into the first window (e.g., into window W1).

[0073] When the control unit 100A detects the second operation (step S203: YES), it issues an instruction to the target source device (step S204). The target source device is the information processing device 10 (e.g., information processing device 10E) that is the source of the mirrored image displayed in the first window (e.g., window W1) where the first object (e.g., object C2a) was dragged.

[0074] In step S204, the control unit 100A instructs the source device to display the second window and to display the first object in the second window. In the example shown in the upper part of Figure 7, when object C2a (an example of the first object) is dragged into window W1 (an example of the first window), the control unit 100A instructs the information processing device 10E, which is the source of the mirrored image displayed in window W1, to display window W2 (an example of the second window) and to display object C2a (an example of the first object) in window W2. As a result, the information processing device 10E displays window W2 with object C2a on screen 142E (see the lower part of Figure 7).

[0075] The second operation is confirmed when parent user A drops object C2a. In step S205, the control unit 100A determines whether the second operation has been confirmed. Until the second operation is confirmed, the display position of window W2 on screen 142E is updated sequentially according to the drag position of object C2a on screen 142A. That is, the control unit 100A, which operates as a display control unit, instructs the update of the display position of window W2 (an example of a second window) on screen 142E (an example of a second screen) based on the drag position of object C2a (an example of a first object). The information processing device 10E updates the display position of window W2 in response to this instruction.

[0076] Once the second operation is confirmed (step S205: YES), the control unit 100A instructs the source device (e.g., the information processing device 10E) to fix the display position of window W2 at the drop position (step S206). That is, the control unit 100A, acting as a display control unit, instructs the display position of window W2 (an example of a second window) on screen 142E (an example of a second screen) to be fixed based on the drop position of object C2a (an example of a first object). The information processing device 10E then fixes the display position of window W2 according to this decision.

[0077] The above is a description of exemplary embodiments of the present disclosure. Embodiments of the present disclosure are not limited to those described above, and various modifications are possible within the scope of the technical idea of ​​the present disclosure. For example, embodiments of the present application include combinations of embodiments explicitly shown in the specification or obvious embodiments as appropriate.

[0078] For example, objects may be assigned a priority. When the first object is dropped into the first window, the objects may be arranged in a layout that reflects their priority. Figure 10 illustrates the layout of objects that reflects their priority.

[0079] In the example in Figure 10, object C3 has a high priority and object C4 has a low priority. In the example in the upper part of Figure 10, parent user A drops object C2a (an example of the first object) in a position that overlaps with both object C3 and object C4. In this case, the display size of object C2a is reduced so that it does not overlap with, for example, the high-priority object C3. Object C2a is displayed in front of, for example, the low-priority object C4. As shown in the lower part of Figure 10, in the screen 142E of the information processing device 10E acting as the source device, window W2 is also displayed with a size and position corresponding to object C2a.

[0080] In the example shown in Figure 10, even when a first object is added, the visibility of the higher-priority object is ensured. In this way, the control unit 100A, which operates as a display control unit, determines the layout of the first object and each second object based on the priority of each second object when the drop position of the first object (for example, object C2a) overlaps with any of the multiple second objects (for example, objects C3 and C4) displayed in window W1 (an example of the first window).

[0081] Object priority is set in advance by, for example, the provider of application 110A. Object priority may also be set according to user operations.

[0082] Although the above embodiment was described using the interior of a vehicle as an example, this embodiment can also be applied to situations where multiple information processing devices 10 are operating, such as working in an office, giving a presentation in a conference room, or playing games in a living room.

[0083] Figure 11 shows an example of a front screen of screen 142A according to Modification 1 of the present disclosure. In the above embodiment, the interior of the vehicle is divided into vertical columns (right column, center column, left column), but the division units are not limited to this. For example, in Modification 1, the interior of the vehicle is divided into horizontal columns. Specifically, in Modification 1, the interior of the vehicle is divided into a first-row area (the area including the passenger seat), a second-row area (the area including the three seats of the second row), and a third-row area (the area including the three seats of the third row).

[0084] For example, in Modification 1, when parent user A swipes their finger horizontally to the right by a predetermined distance in area 200ce within screen 142A corresponding to the second row area, group window GW3 is displayed on screen 142A. The group window GW3 displays components 210cC to 210cE corresponding to information processing devices 10C to 210cE installed on each seat of the second row. Parent user A can understand what content the child user is viewing, as in the above embodiment. Parent user A can easily operate the screen 142 of the child user's information processing device 10, which is physically out of reach, via the information processing device 10A, as in the above embodiment.

[0085] For example, if there is only one information processing device 10 installed in the partitioned area, the display of the group window may be omitted. Figure 12 shows side by side an example of the display screens of screen 142A and screen 142B according to Modification 2 of this disclosure. In the example in Figure 12, parent user A swipes their finger a predetermined distance horizontally to the left from the right edge (in other words, the passenger side edge) of screen 142A.

[0086] In area R1, only the information processing device 10B corresponding to the passenger seat is installed. In this case, window W3 appears as if being pulled horizontally to the left from the right edge of the screen. Window W3 displays a mirrored image of the information processing device 10B, which is the source of the shared information, in a windowed format. Parent user A can avoid the trouble of selecting the small window 220 from within the group window. [Explanation of Symbols]

[0087] 1: In-vehicle systems 10: Information Processing Device 100: Control Unit 110A: Application 140:Display section 142: Screen

Claims

1. An information processing device that is communicatively connected to source devices installed in each partitioned area of ​​a predetermined space, A display unit having a first screen, A display control unit that displays an image on the first screen, The system includes an acquisition unit that acquires a shared source image which is a display image of the second screen of the source device, The display control unit, When a first selection operation to select the partition area is detected, the shared source images displayed on the second screen of each source device installed in the selected partition area are displayed on the first screen. Information processing device.

2. A detection unit for detecting a source device in use from among a plurality of source devices installed in the predetermined space, A partition section that demarcates the partition area from the predetermined space based on the detected installation position of the source device in use, Equipped with, The information processing apparatus according to claim 1.

3. The aforementioned predetermined space is the interior of the mobile body. The detection unit is The occupants in the aforementioned room are detected, The source device corresponding to the seat detected by the occupant is considered to be the source device currently in use. The information processing apparatus according to claim 2.

4. The display control unit, The shared source images of each source device installed in the partition area selected by the first selection operation are displayed at a first magnification. When a second selection operation is detected in which one of the shared source images displayed on the second screen of each source device installed in the partition area selected in the first selection operation is selected, the selected shared source image is displayed at a second magnification greater than the first magnification. An information processing apparatus according to any one of claims 1 to 3.

5. An information processing method performed by an information processing device that is communicatively connected to source devices installed in each partitioned area of ​​a predetermined space, The shared source image, which is the display image of the second screen of the aforementioned source device, is obtained. When a first selection operation for selecting the partition area is detected, the shared source images displayed on the second screen of each source device installed in the selected partition area are displayed on the first screen of the information processing device. Information processing methods.

6. From among the multiple source devices installed in the predetermined space, the source device currently in use is detected. The partition area is demarcated from the predetermined space based on the detected installation position of the source device in use. The information processing method according to claim 5.

7. The aforementioned predetermined space is the interior of the mobile body. The occupants in the aforementioned room are detected, The source device corresponding to the seat detected by the occupant is considered to be the source device currently in use. The information processing method according to claim 6.

8. The shared source images of each source device installed in the partition area selected by the first selection operation are displayed at a first magnification. When a second selection operation is detected in which one of the shared source images displayed on the second screen of each source device installed in the partition area selected in the first selection operation is selected, the selected shared source image is displayed at a second magnification greater than the first magnification. The information processing method according to any one of claims 5 to 7.

Images