Information processing apparatus, information processing system, information processing method, and carrier means for point cloud modification

Abstract

An information processing apparatus includes a creation unit configured to create a display screen including a first display region that displays a three-dimensional point cloud and a second display region that displays information on an article, and a processing unit configured to move a subset point cloud of the three-dimensional point cloud relative to rest of the three-dimensional point cloud other than the subset point cloud. The creation unit creates the display screen displaying, in the first display region, the subset point cloud that has been moved and displaying, in the second display region, the information on the article associated with the subset point cloud before the movement of the subset point cloud in association with the subset point cloud after the movement.

Description

FN202502325[DESCRIPTION][Title of Invention]INFORMATION PROCESSING APPARATUS, INFORMATION PROCESSING SYSTEM, INFORMATION PROCESSING METHOD, AND CARRIER MEANS[Technical Field]

[0001] The present disclosure relates to an information processing apparatus, an information processing system, an information processing method, and carrier means.[Background Art]

[0002] In the related art, there is a so-called digital twin technique for acquiring three-dimensional information of a real space as digital data with a laser scanner, Light Detection and Ranging (LiDAR), photogrammetry, or the like, and reproducing a virtual space equivalent to the physical space on a computer based on the three-dimensional information. In the related art, an operation of an article included in created drawing data has recently been checked by representing the article in a virtual space generated by a computer and moving the article or causing the article to operate in the virtual space.PTL 1 describes a drawing data creation method for selecting a subset of a point cloud of a target object as a virtual object element, copying the selected object element, and pasting or pasting and moving the copied object element to arrange the copied object element in a virtual space.[Citation List][Patent Literature]

[0003] [PTL 1]lapanese Patent No. 6005317[Summary of Invention][Technical Problem]

[0004] In the technique of the related art, information on an article associated with the copied subset (i.e., subset point cloud) of the point cloud is not associated with the subset point cloud after the movement. Thus, the information on the article is laboriously manually associated with the subset point cloud after the movement.[Solution to Problem]

[0005] The present disclosure described herein provides an information processing apparatus including a creation unit that creates a display screen including a first display region that displays a three-dimensional point cloud and a second display region that displays information on an article, and a processing unit that moves a subset point cloud of the three-dimensional point cloud relative to rest of the three-dimensional point cloud other than theFN202502325subset point cloud. The creation unit creates the display screen displaying, in the first display region, the subset point cloud that has been moved and displaying, in the second display region, the information on the article associated with the subset point cloud before the movement of the subset point cloud in association with the subset point cloud after the movement.The present disclosure described herein provides an information processing system including a creation unit that creates a display screen including a first display region that displays a three-dimensional point cloud and a second display region that displays information on an article, a display control unit that displays the display screen on a display, an accepting unit that accepts an operation to move a subset point cloud of the three-dimensional point cloud displayed in the first display region relative to rest of the three-dimensional point cloud other than the subset point cloud, and a processing unit that moves the subset point cloud relative to the rest of the three-dimensional point cloud, based on the accepted operation. The creation unit creates the display screen displaying, in the first display region, the subset point cloud that has been moved and displaying, in the second display region, the information on the article associated with the subset point cloud before the movement of the subset point cloud in association with the subset point cloud after the movement.The present disclosure described herein provides an information processing method to be executed by an information processing apparatus, including creating a display screen including a first display region that displays a three-dimensional point cloud and a second display region that displays information on an article, and moving a subset point cloud of the three-dimensional point cloud relative to rest of the three-dimensional point cloud other than the subset point cloud. The creating includes creating the display screen displaying, in the first display region, the subset point cloud that has been moved and displaying, in the second display region, the information on the article associated with the subset point cloud before the movement of the subset point cloud in association with the subset point cloud after the movement.The present disclosure described herein provides carrier means carrying computer readable code for controlling a computer system to carry out the information processing method described above.[Advantageous Effects of Invention]

[0006] According to one aspect of the present disclosure, even when a subset point cloud of a point cloud is moved relative to the rest of the point cloud, information on an article associated with the subset point cloud and information on the subset point cloud after the movement are stored in association with each other.[Brief Description of Drawings]

[0007] FN202502325A more complete appreciation of embodiments of the present disclosure and many of the attendant advantages and features thereof can be readily obtained and understood from the following detailed description with reference to the accompanying drawings.FIG. l is a general arrangement diagram illustrating an example of an information processing system according to a first embodiment.FIG. 2 is a hardware configuration diagram illustrating an example of a terminal apparatus and a management server according to the first embodiment.FIG. 3 is a functional block diagram illustrating an example of the information processing system according to the first embodiment.FIG. 4 is a diagram illustrating an example of a display screen of a three-dimensional (3D) viewer.FIG. 5 is a diagram illustrating another example of the display screen of the 3D viewer.FIG. 6 is a diagram illustrating an example of equipment information.FIG. 7 is a diagram illustrating an example of the display screen created by a creation unit. FIG. 8 is a diagram illustrating another example of the display screen created by the creation unit.FIG. 9 is a diagram illustrating another example of the display screen created by the creation unit.FIG. 10 is a diagram illustrating another example of the display screen created by the creation unit.FIG. 11 is a flowchart illustrating an example of a procedure of an association process according to the first embodiment.FIG. 12 is a diagram illustrating an example of the display screen in which a subset point cloud is identified.FIG. 13 is a diagram illustrating an example of the display screen in which the subset point cloud is translated.FIG. 14 is a diagram illustrating an example of the display screen in which the subset point cloud is rotated.FIG. 15 is a flowchart illustrating an example of a procedure of a movement process of moving a subset point cloud according to the first embodiment.FIG. 16 is a functional block diagram illustrating an example of an information processing system according to a second embodiment.FIG. 17 is a diagram illustrating another example of the display screen created by the creation unit.FIG. 18 is a flowchart illustrating an example of a procedure of an association process according to the second embodiment.The accompanying drawings are intended to depict embodiments of the present disclosure and should not be interpreted to limit the scope thereof. The accompanying drawings are not to be considered as drawn to scale unless explicitly noted. Also, identical or similar reference numerals designate identical or similar components throughout the several views.FN202502325[Description of Embodiments]

[0008] In describing embodiments illustrated in the drawings, specific terminology is employed for the sake of clarity. However, the disclosure of this specification is not intended to be limited to the specific terminology so selected and it is to be understood that each specific element includes all technical equivalents that have a similar function, operate in a similar manner, and achieve a similar result.Referring now to the drawings, embodiments of the present disclosure are described below. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0009] First EmbodimentFIG. l is a general arrangement diagram illustrating an example of an information processing system 1 according to a first embodiment.The information processing system 1 includes a terminal apparatus 10 and a management server 40.

[0010] The management server 40 is an example of an information processing apparatus. The management server 40 performs one or more information processing processes on three-dimensional information such as a three-dimensional point cloud or a mesh object.

[0011] A three-dimensional point cloud is a collection of coordinate points, in directions such as X, Y, and Z directions, corresponding to measurement points on the surface of an object obtained when a certain space in which the object is present is measured using a laser scanner LS, for example. A coordinate point is represented as (1, 3, 5), for example. Each coordinate point may additionally have color information. The color information added may be red, green, and blue (RGB) values of each coordinate point.

[0012] The three-dimensional point cloud may also be referred to as a point cloud. Point cloud data is data that allows the three-dimensional point cloud to be handled by a computer as a collection of coordinate points in a virtual 3D space.

[0013] An example of measuring a three-dimensional point cloud using the LS is presented above. However, the three-dimensional point cloud may be measured using another optical measuring means or a mechanical measuring means. Examples of another optical measuring means include a method using a stereo camera and a method using visual simultaneous localization and mapping (SLAM).

[0014] Examples of the three-dimensional information may include a digital model (3D model) or 3D computer-aided design (CAD) model of a building or equipment inside the building used inFN202502325Building Information Modeling (BIM). Equipment (such as a device, an installation, or an apparatus) is an example of an article.

[0015] The terminal apparatus 10 and the management server 40 communicate with each other via a communication network 20. The communication network 20 includes the Internet, a mobile communication network, and a local area network (LAN), for example. The communication network 20 may include, in addition to a wired communication network, a wireless communication network that is compliant with 3rd Generation (3G), Worldwide Interoperability for Microwave Access (WiMAX), Long Term Evolution (LTE), or 5th Generation (5G). The terminal apparatus 10 may perform communication using short-range communication technology such as Near Field Communication (NFC®).

[0016] Hardware ConfigurationFIG. 2 is a hardware configuration diagram illustrating an example of the terminal apparatus 10 and the management server 40 according to the first embodiment. The hardware components of the terminal apparatus 10 are denoted by reference numerals in 100 series. The hardware components of the management server 40 are denoted by reference numerals in 400 series.

[0017] The terminal apparatus 10 includes a central processing unit (CPU) 101, a read-only memory (ROM) 102, a random access memory (RAM) 103, a hard disk drive (HDD) 104, an HDD controller 105, a display interface (I / F) 106, a communication I / F 107, a sensor I / F 108, an audio input / output I / F 109, an input I / F 110, a media I / F 111, a digital versatile disk rewritable (DVD-RW) drive 112, and a bus line 113.

[0018] The CPU 101 controls the overall operation of the terminal apparatus 10. The ROM 102 stores a program used for booting the CPU 101. The RAM 103 is used as a work area for the CPU 101. The HDD 104 stores various types of data such as a program. The HDD controller 105 controls reading or writing of various types of data from or to the HDD 104 under control of the CPU 101.

[0019] The display I / F 106 is an interface for connecting a display 106a to the terminal apparatus 10. The display 106a displays various types of information such as a cursor, a menu, a window, text, or an image. The communication I / F 107 is an interface for performing data communication via the communication network 20. The sensor I / F 108 is an interface that couples various sensors to the terminal apparatus 10.

[0020] The audio input / output I / F 109 is an interface for connecting a speaker 109a and a microphone 109b to the terminal apparatus 10. The input I / F 110 is an interface for connecting input devices such as a keyboard 110a and a mouse 110b to the terminal apparatusFN20250232510. The keyboard 110a is an example of an input means provided with multiple keys used for inputting characters, numerical values, and various instructions. The mouse 110b is an example of an input means for selecting or executing various instructions, selecting a target for processing, or moving a cursor.

[0021] The media I / F 111 controls reading or writing (storing) of data from or to a recording medium Illa such as a flash memory.

[0022] The DVD-RW drive 112 controls reading or writing of various types of data from or to a DVD-RW 112a, which is an example of a removable recording medium. The terminal apparatus 10 may further include a component for controlling reading or writing (storing) of data from or to an external personal computer (PC) or external device connected with a cable or connected wirelessly by wireless fidelity (Wi-Fi®).

[0023] The management server 40 includes a CPU 401, a ROM 402, a RAM 403, an HDD 404, an HDD controller 405, a display I / F 406, a communication I / F 407, a sensor I / F 408, an audio input / output I / F 409, an input I / F 410, a media I / F 411, a DVD-RW drive 412, and a bus line 413. Since the components of the management server 40 are substantially the same as the respective components of the terminal apparatus 10 described above (i.e., the CPU 101, the ROM 102, the RAM 103, the HDD 104, the HDD controller 105, the display I / F 106, the communication I / F 107, the sensor I / F 108, the audio input / output I / F 109, the input I / F 110, the media I / F 111, the DVD-RW drive 112, and the bus line 113), a description thereof is omitted.

[0024] The DVD-RW drive 112 (412) may be replaced with a DVD-R drive or any other memory. In one embodiment, each of the terminal apparatus 10 and the management server 40 is implemented by a single computer. In another embodiment, each of the terminal apparatus 10 and the management server 40 is implemented by multiple computers to which the components (functions, means, or storages) are distributed as desired.

[0025] FIG. 3 is a functional block diagram illustrating an example of the information processing system 1 according to the first embodiment.

[0026] Functional Configuration of Terminal ApparatusAs illustrated in FIG. 3, the terminal apparatus 10 includes a transmitting and receiving unit 11, an accepting unit 12, a display control unit 14, and a storing and reading unit 19. These units are functions that are implemented by or means that are caused to function by any of the components illustrated in FIG. 2 operating in accordance with the instructions from the CPU 101 based on a program loaded to the RAM 103 from the HDD 104. The terminal apparatusFN20250232510 also includes a storage unit 1000 implemented by the RAM 103 and the HDD 104 illustrated in FIG. 2.

[0027] Functional Components of Terminal ApparatusFunctional components of the terminal apparatus 10 will be described.

[0028] The transmitting and receiving unit 11 is an example of a reception means and is implemented by instructions from the CPU 101 and by the communication I / F 107 illustrated in FIG. 2. The transmitting and receiving unit 11 transmits and receives various types of data (or information) to and from other terminals, apparatuses, or systems via the communication network 20.

[0029] The accepting unit 12 is an example of an accepting means and is implemented by instructions from the CPU 101 and by the keyboard 110a and the mouse 110b illustrated in FIG. 2. The accepting unit 12 receives various inputs from a user.

[0030] The display control unit 14 is an example of a display control means and is implemented by instructions from the CPU 101 illustrated in FIG. 2. The display control unit 14 causes the display 106a, which is an example of a display, to display various images or screens.

[0031] For example, the terminal apparatus 10 uses an application (i.e., 3D viewer) for visualizing three-dimensional information to display the three-dimensional information. The 3D viewer is a web application used on a web browser, a PC application that operates on a PC, or a smartphone application that operates on a smartphone, for example.

[0032] The three-dimensional information is, for example, information uploaded to a cloud service or information stored on a device of the terminal apparatus 10 such as a PC or a smartphone. The terminal apparatus 10 can visualize the three-dimensional information on a 3D viewer regardless of the type of the 3D viewer or the location of the three-dimensional information. The three-dimensional information may be data in any format that can be segmented into elements, such as a mesh or point cloud.

[0033] FIG. 4 is a diagram illustrating an example of a display screen of the 3D viewer. The information processing system 1 uses the three-dimensional information and image information obtained by capturing a real space in advance to visualize the three-dimensional information and reproduce the real space with the 3D viewer. That is, the information processing system 1 implements a so-called digital twin. FIG. 4 illustrates a display screen 300 including a first display region 310 that displays an any -viewpoint image (i.e., an image of a point cloud viewed from any viewpoint) generated by projecting a three-dimensional point cloud acquired inside a machine room onto a two-dimensional plane.FN202502325

[0034] FIG. 5 is a diagram illustrating another example of the display screen of the 3D viewer. In this example, FIG. 5 illustrates the first display region 310 that displays a bird's-eye view image (i.e., an image of the entire machine room in the bird's-eye view) generated by projecting the three-dimensional point cloud of the entire machine room in FIG. 4 onto a two-dimensional plane. The user can view the visualized three-dimensional information while switching the display between the any-viewpoint image illustrated in FIG. 4 and the bird's-eye view image illustrated in FIG. 5. The user performs an input operation via the accepting unit 12 to change the viewpoint of the any-viewpoint image or the angle of the bird's-eye view image. Thus, the user can check the content of the three-dimensional information at various angles or from various viewpoints. As described above, the 3D viewer enables checking and bird's-eye display of an image of any image-captured point inside the building and a walkthrough inside the three-dimensional space.

[0035] The storing and reading unit 19 is an example of a storage control unit. The storing and reading unit 19 is implemented by instructions from the CPU 101 and by the HDD controller 105, the media I / F 111, and the DVD-RW drive 112 illustrated in FIG. 2 and by an external PC or an external device. The storing and reading unit 19 stores various types of data in the storage unit 1000, the recording medium Illa, the DVD-RW 112a, and the external PC or the external device. The storing and reading unit 19 reads various types of data from the storage unit 1000, the recording medium Illa, the DVD-RW 112a, and the external PC or the external device. The storage unit 1000, the recording medium Illa, the DVD-RW 112a, and the external PC or the external device are each an example of a storage unit.

[0036] Functional Configuration of Management ServerAs illustrated in FIG. 3, the management server 40 includes a transmitting and receiving unit 41, a processing unit 43, a creation unit 47, and a storing and reading unit 49. These units are functions that are implemented by or means that are caused to function by any of the components illustrated in FIG. 2 operating in accordance with the instructions from the CPU 401 based on a program loaded to the RAM 403 from the HDD 404. The management server 40 further includes a storage unit 4000 implemented by the HDD 404 illustrated in FIG. 2. The storage unit 4000 is an example of a storage means.

[0037] Functional Components of Management ServerFunctional components of the management server 40 will be described. The management server 40 may be implemented by multiple computers such that functions thereof are distributed to the multiple computers. The management server 40 is described as a server computer residing in a cloud environment but may be a server residing in an on-premises environment.

[0038] FN202502325The transmitting and receiving unit 41 is an example of a transmission means. The transmitting and receiving unit 41 is implemented by instructions from the CPU 401 and by the communication I / F 407 illustrated in FIG. 2. The transmitting and receiving unit 41 transmits and receives various types of data (or information) to and from other terminals, apparatuses, or systems via the communication network 20.

[0039] The transmitting and receiving unit 41 receives equipment information. The received equipment information is stored in an article information management database (DB) 4002 via the storing and reading unit 49. The equipment information is information on equipment inside a building. The equipment information includes, for example, information indicating an identification (ID) of equipment, an installed position of the equipment, a name of the equipment, and a maintenance timing. The equipment information is an example of information on an article.

[0040] FIG. 6 is a diagram illustrating an example of the equipment information. As illustrated in FIG. 6, the equipment information includes various types of information on equipment (e.g., information on an equipment ID, a room code, an equipment name, and an inspection date and time). A building includes many pieces of equipment, for each of which the statutory inspection and maintenance timings are set. Information on the "last inspection date" and the "next inspection date" in FIG. 6 are used for managing the inspection and maintenance timings of each piece of equipment.

[0041] A "three-dimensional position" in FIG. 6 represents the coordinates (i.e., position information) of the installed position of equipment as (X, Y, Z) format, for example. The three-dimensional position allows the user to grasp the installed position of the equipment to be maintained. A "captured image" is, for example, a captured image of equipment obtained when the equipment is installed. In the example in FIG. 6, a file name of the captured image is stored. The equipment information may store link information to the captured image instead of the file name of the captured image.

[0042] The management server 40 may generate the equipment information. For example, an administrator operates a keyboard 410a or a mouse 410b to add or edit the equipment information. The added or edited equipment information is stored in the article information management DB 4002 via the storing and reading unit 49. The file of the equipment information may be in any format that can describe information of items. Examples of the file format to be used include Excel format or comma-separated values (CSV) format.

[0043] The transmitting and receiving unit 41 receives the equipment information in a file format or receives the equipment information using an application programming interface (API) from a database of an external system. The item names of the equipment information illustrated inFN202502325FIG. 6 are an example, and different items may be used in accordance with the state of a building in which the equipment is installed.

[0044] The processing unit 43 is implemented by instructions from the CPU 401 illustrated in FIG. 2. The processing unit 43 performs various processes including an association process of associating position information and equipment information with each other (described later) and a movement process of moving a subset (i.e., subset point cloud) of a three-dimensional point cloud (described later).

[0045] The creation unit 47 is implemented by instructions from the CPU 401 illustrated in FIG. 2. The creation unit 47 creates the display screen 300 to be displayed on the 3D viewer, for example.

[0046] The storing and reading unit 49 is an example of a storage control unit. The storing and reading unit 49 is implemented by instructions from the CPU 401 and by the HDD controller 405, the media I / F 411, and the DVD-RW drive 412 illustrated in FIG. 2 and by an external PC or an external device. The storing and reading unit 49 stores various types of data in the storage unit 4000, a recording medium 411a, a DVD-RW 412a, and the external PC or the external device. The storing and reading unit 49 reads various types of data from the storage unit 4000, the recording medium 411a, the DVD-RW 412a, and the external PC or the external device. The storage unit 4000, the recording medium 411a, the DVD-RW 412a, and the external PC or the external device are each an example of a storage unit.

[0047] The storage unit 4000 includes a three-dimensional point cloud management DB 4001 and the article information management DB 4002. The storage unit 4000 may be implemented by multiple servers. For example, the three-dimensional point cloud management DB 4001 may be built in a first server and the article information management DB 4002 may be built in a second server.

[0048] The screen created by the creation unit 47 and the association process of associating position information and equipment information with each other performed by the processing unit 43 will be described with reference to FIGs. 7 to 11.

[0049] FIG. 7 is a diagram illustrating an example of the display screen 300 created by the creation unit 47. The display screen 300 illustrated in FIG. 7 includes the first display region 310 and a second display region 320. The first display region 310 displays an image generated by projecting a three-dimensional point cloud onto a two-dimensional plane. The second display region 320 displays equipment information. For example, the user selects an equipment information display function of the 3D viewer while viewing the two-dimensional image in the first display region 310 as illustrated in FIG. 4 or 5 to add the second display region 320 toFN202502325the display screen 300. The first display region 310 is an example of a first display region. The second display region 320 is an example of a second display region.

[0050] The second display region 320 illustrated in FIG. 7 displays the equipment name, the model number, and the last inspection date among pieces of information of the equipment information. Limiting the number of items to be displayed in this manner allows pieces of equipment information of multiple pieces of equipment to be displayed in the second display region 320.

[0051] FIG. 8 is a diagram illustrating another example of the display screen 300 created by the creation unit 47. Differences between FIGs. 8 and 7 are that the second display region 320 displays equipment information of equipment selected from among multiple pieces of equipment and additional display items. The user uses the accepting unit 12 to select desired equipment from among the multiple pieces of equipment whose equipment information is displayed in the second display region 320 illustrated in FIG. 7 to display equipment information of the selected equipment in the second display region 320.

[0052] In FIG. 8, the second display region 320 displays, as additional items, an installation date and inspection-related laws and regulations (e.g., names of laws and regulations serving as a basis for the inspection). The user uses the accepting unit 12 to edit the content of each item displayed in the second display region 320.

[0053] The second display region 320 illustrated in FIG. 8 displays a designate position button 321. The designate position button 321 is displayed when the position information is not associated with the equipment information of the selected equipment. When the equipment information is, for example, received from spreadsheet software or equipment ledger management software, the position information is not associated with the equipment information. That is, the equipment information does not include the item "three-dimensional position". In the present embodiment, in such a case, the position information and the equipment information are associated with each other in the following manner.

[0054] The user uses the accepting unit 12 (such as the mouse 110b) to press the designate position button 321 to display a position pointer 311 in the first display region 310. The user uses the accepting unit 12 to move the position pointer 311 to a desired two-dimensional position, and then presses the designate position button 321 to designate the two-dimensional position.

[0055] The processing unit 43 obtains position information indicating the three-dimensional position of the equipment from information of the three-dimensional point cloud projected onto an area around the designated two-dimensional position, and associates the position information and the equipment information with each other. For example, the processing unit 43 obtains,FN202502325as the position information indicating the three-dimensional position of the equipment, a position of the centroid of the three-dimensional point cloud projected to the area around the designated two-dimensional position.

[0056] The storing and reading unit 49 stores associated information obtained by the processing unit 43 in the storage unit 4000 or any other memory. The associated information includes the coordinates of the installed position added as the three-dimensional position of the equipment information, equipment information written to the point cloud data indicated by the position information, and link information to an equipment information file that is written in the point cloud data.

[0057] When the position information is associated with the equipment information of the selected equipment, the creation unit 47 creates the display screen 300 that displays, in the first display region 310, the three-dimensional point cloud of the area around the three-dimensional position corresponding to the position information. This allows the user to check the selected equipment and the three-dimensional point cloud of the area around the selected equipment. In this case, the second display region 320 may display a change position button instead of the designate position button 321. The user presses the change position button to display the position pointer 311, moves the position pointer 311, and then presses the change position button to change (edit) the existing position information.

[0058] When the position information is not associated with the equipment information, the creation unit 47 may use the room code of the equipment information to create the display screen 300 that displays, in the first display region 310, a three-dimensional point cloud of a position near the equipment. This allows the user to easily designate the two-dimensional position of the equipment. Information such as the room code is stored in the three-dimensional point cloud when the three-dimensional point cloud is acquired, for example.

[0059] When the position information is not associated with the equipment information and the room code is not included in the equipment information, the processing unit 43 may add a code representing a room including the two-dimensional position designated by the user to the equipment information as the room code along with the association process. This allows the user to add the room code of the equipment information by designating the two-dimensional position of the equipment.

[0060] FIG. 9 is a diagram illustrating another example of the display screen 300 created by the creation unit 47. A difference between FIGs. 9 and 7 is that the equipment information displayed in the second display region 320 includes a position icon 322. The position icon 322 indicates that the position information is associated with the equipment information. In the example in FIG. 9, the position icon 322 is displayed for equipment information of air-FN202502325conditioning equipment. This indicates that the equipment information of the air-conditioning equipment is already associated with the position information. On the other hand, the position icon 322 is not displayed for equipment information of piping. This indicates that the equipment information of the piping is not associated with the position information yet.

[0061] As described above, the user can easily recognize whether equipment information is associated with position information based on whether the position icon 322 is displayed. The position icon 322 is an example of a distinctive image indicating that the position information and the equipment information are associated with each other.

[0062] FIG. 10 is a diagram illustrating another example of the display screen 300 created by the creation unit 47. A difference between FIGs. 10 and 9 is that the equipment information displayed in the second display region 320 includes a thumbnail image 323. The creation unit 47 uses data of the captured image included in the equipment information to generate the thumbnail image 323, so that the thumbnail image 323 is displayed in the second display region 320.

[0063] When the equipment information is not associated with the position information, the creation unit 47 generates, as the thumbnail image 323, an image prepared in advance for the equipment name, for example, so that the thumbnail image 323 is displayed in the second display region 320. In this case, the thumbnail image 323 is, for example, a text image including the equipment name, an icon image schematically illustrating the equipment, or an icon image including a general photo of the equipment.

[0064] The thumbnail image 323 displayed in this manner allows the user to easily select desired equipment from the second display region 320.

[0065] FIG. 11 is a flowchart illustrating an example of a procedure of the association process according to the first embodiment.

[0066] In step SI 00, the transmitting and receiving unit 41 receives equipment information. In step S 101, the creation unit 47 creates the display screen 300 including the first display region 310 and the second display region 320. At this time, the second display region 320 displays multiple pieces of equipment information received by the transmitting and receiving unit 41. The display control unit 14 displays the created display screen 300 on the display 106a.Thereafter, the display control unit 14 similarly displays the created display screen 300 on the display 106a.

[0067] In step SI 02, the transmitting and receiving unit 41 receives desired equipment selected in the second display region 320 from the user. In step S103, the creation unit 47 creates the displayFN202502325screen 300 that displays detailed information on the selected equipment in the second display region 320.

[0068] In step SI 04, the processing unit 43 determines whether the selected equipment information is associated with the position information. When the selected equipment information is associated with the position information (step S104: Yes), the association process ends. When the selected equipment information is not associated with the position information (step S104: No), step SI 05 is performed. In step SI 05, the transmitting and receiving unit 41 receives the two-dimensional position of the equipment designated by the user. Specifically, the user presses the designate position button 321 to display the position pointer 311, moves the position pointer 311 to a desired position, and presses the designate position button 321 to designate the two-dimensional position. The transmitting and receiving unit 41 receives the two-dimensional position thus designated.

[0069] In step SI 06, the processing unit 43 associates the position information and the equipment information with each other based on the designated two-dimensional position. The storing and reading unit 49 stores the associated information in the storage unit 4000 or any other memory.

[0070] The functions of the management server 40 illustrated in FIG. 3 may be integrated into the terminal apparatus 10, and the terminal apparatus 10 may perform the above-described process of the management server 40.

[0071] The movement process of moving a subset (i.e., a subset point cloud) of a three-dimensional point cloud performed by the processing unit 43 will be described with reference to FIGs. 12 to 15.

[0072] FIG. 12 is a diagram illustrating an example of the display screen 300 in which a subset point cloud is identified. Differences between FIGs. 12 and 10 are that the second display region 320 further includes a designate equipment button 325 and a confirm movement button 326. In this example, the first display region 310 displays three-dimensional point clouds corresponding to a chilled water pump, air-conditioning equipment, and piping installed on the basement floor.

[0073] The user uses the accepting unit 12 (such as the mouse 110b) to press the designate equipment button 325 to display a pointer in the first display region 310. The user uses the accepting unit 12 to move the pointer and press the designate equipment button 325 to designate the position where the desired equipment is located in the first display region 310.

[0074] FN202502325The processing unit 43 detects equipment located near the position designated with the designate equipment button 325. The detected equipment is, for example, equipment included in a bounding box including a point cloud corresponding to the equipment. For example, the processing unit 43 uses artificial intelligence (Al) to detect the bounding box based on the three-dimensional point cloud.

[0075] The detected bounding box is a rectangular parallelepiped including equipment located around the designated position. In the example of FIG. 12, the position of the chilled water pump is designated as equipment, and a bounding box 313 including the chilled water pump is detected. FIG. 12 is a front view of the rectangular parallelepiped representing the bounding box 313.

[0076] The Al used by the processing unit 43 in a three-dimensional object detection process is trained by machine learning on point cloud data, which serves as input, and a bounding box and the type of equipment corresponding to the bounding box, which serve as output (ground truth data).

[0077] The bounding box and the type of equipment used in machine learning are each detectable by object detection and instance segmentation, for example. By using feature quantities (such as positional relationships among coordinates of respective points of a point cloud, colors, normals, and reflection intensities) as input data in common, machine learning can be performed on Al that detects bounding boxes and Al that detects classes as output data.Instance segmentation is an example of region segmentation.

[0078] The processing unit 43 identifies a subset point cloud included in the detected bounding box including the equipment. In FIG. 12, the processing unit 43 identifies, as the subset point cloud, a point cloud included in the bounding box 313.

[0079] FIG. 13 is a diagram illustrating an example of the display screen 300 in which the subset point cloud is translated. A difference between FIGs. 13 and 12 is that the point cloud (i.e., the subset point cloud) included in the bounding box 313 is translated toward the front.

[0080] The user uses the accepting unit 12 to select and drag the bounding box 313 in FIG. 12 to translate the bounding box 313 to the position in FIG. 13. In response to the translation of the bounding box 313, the processing unit 43 translates the subset point cloud relative to the rest of the point cloud.

[0081] FIG. 14 is a diagram illustrating an example of the display screen 300 in which the subset point cloud is rotated. A difference between FIGs. 14 and 13 is that the point cloud (i.e., theFN202502325subset point cloud) included in the bounding box 313 is rotated rightward by 120 degrees on a horizontal plane in FIG. 14.

[0082] The user uses the accepting unit 12 to select the bounding box 313 in FIG. 13 and uses a rotation handle to rotate the bounding box 313 as illustrated in FIG. 14. In response to the rotation of the bounding box 313, the processing unit 43 rotates the subset point cloud relative to the rest of the point cloud.

[0083] In the present embodiment, "movement" collectively refers to translation and rotation. The processing unit 43 identifies, as a subset point cloud, a point cloud included in a bounding box in the above. However, the subset point cloud is not limited to such a point cloud. For example, the processing unit 43 may use a segmentation mask obtained by instance segmentation to identify a subset point cloud.The processing unit 43 may identify, as the subset point cloud, a point cloud in a region designated by the user with the accepting unit 12 in the first display region 310. The segmentation mask is an example of a segmented region.

[0084] In FIGs. 13 and 14, in response to the user pressing the confirm movement button 326, the processing unit 43 associates equipment information of the detected equipment and the position of the subset point cloud with each other, and the storing and reading unit 49 stores the associated pieces of information in the storage unit 4000 or any other memory. The information that the processing unit 43 associates with the equipment information is not limited to the information on the position of the moved subset point cloud. The processing unit 43 may associate the equipment information with at least one of a point included in the subset point cloud, three-dimensional coordinates included in the region corresponding to the subset point cloud, the above-described bounding box, and the above-described segmented region, for example. The processing unit 43 may further associate the equipment information associated with a subset point cloud before the movement, with three-dimensional coordinates included in a region corresponding to the subset point cloud before the movement.

[0085] FIG. 15 is a flowchart illustrating an example of a procedure of the movement process of moving a subset point cloud according to the first embodiment.

[0086] In step SI 50, the processing unit 43 determines whether there is a user operation to designate equipment via the transmitting and receiving unit 41. When there is a user operation to designate equipment (step SI 50: Yes), the processing unit 43 detects equipment at the designated position in step SI 51, and identifies a subset point cloud included in a bounding box of the detected equipment in step SI 52. When there is no user operation to designate equipment (step SI 50: No), the process ends.

[0087] FN202502325In step SI 53, the processing unit 43 determines whether there is a user operation to move the bounding box 313. In response to a user operation to move the bounding box 313 (step SI 53: Yes), the processing unit 43 moves the identified subset point cloud in step SI 54. When there is no user operation to move the bounding box 313 (step SI 53: No), the process returns to step SI 53 in which the processing unit 43 waits for the movement operation. When there is no movement operation within a predetermined time, the processing unit 43 may end the process. In response to a cancel button being pressed, the processing unit 43 may end the process.

[0088] In response to a user operation to confirm the movement operation (step SI 55: Yes), the process proceeds to step SI 56. On the other hand, when there is no operation to confirm the movement operation (step SI 55: No), the process returns to step SI 55 in which the processing unit 43 waits for confirmation of the movement operation. When there is no confirmation of the movement operation within a predetermined time, the processing unit 43 may end the process. In response to a cancel button being pressed, the processing unit 43 may end the process.

[0089] In step SI 56, when the detected equipment has equipment information (step SI 56: Yes), the creation unit 47 creates the display screen 300 that continuously displays the equipment information in the second display region 320 to continuously display the equipment information in step SI 57. That is, the creation unit 47 creates the display screen 300 that displays the equipment information associated with the subset point cloud in association with the subset point cloud after the movement in the second display region 320.

[0090] In step SI 58, the processing unit 43 associates the equipment information and the position of the subset point cloud with each other, and the storing and reading unit 49 stores the associated pieces of information in the storage unit 4000 or any other memory.

[0091] On the other hand, when the detected equipment does not have the equipment information (step SI 56: No), the process ends.

[0092] According to the present embodiment, even when a subset point cloud is moved relative to the rest of the point cloud in the first display region 310, the equipment information (i.e., information on an article) associated with the subset point cloud is displayed in association with the subset point cloud after the movement in the second display region 320. The equipment information associated with the subset point cloud is stored in association with information on the subset point cloud after the movement. This can save the user time to associate the equipment information with the subset point cloud after the movement.

[0093] Second EmbodimentFN202502325In a second embodiment, a trained Al model is used to display candidates for a two-dimensional position of equipment in the second display region 320. Description of part of the second embodiment similar to that of the first embodiment will be omitted, and part of the second embodiment different from that of the first embodiment will be described below.

[0094] FIG. 16 is a functional block diagram illustrating an example of the information processing system 1 according to the second embodiment. A difference between FIGs. 16 and 3 is that the storage unit 4000 of the management server 40 further includes a trained model 4003. When multiple servers implement the storage unit 4000, the trained model 4003 may be built in any one of the multiple servers. For example, the trained model 4003 may be built in either the first server or the second server.

[0095] The trained model 4003 illustrated in FIG. 16 is an Al model (i.e., machine learning model) trained by machine learning on training data including equipment information and a two-dimensional image. The two-dimensional image used in machine learning is, for example, an image of equipment indicated by the equipment information.

[0096] Machine learning is a technology that allows a computer to acquire a human-like learning ability. Machine learning refers to a technology that allows a computer to autonomously generate an algorithm for determination such as data identification from training data loaded in advance and applies the generated algorithm to new data to make a prediction. The machine learning method may be any one of supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and deep learning, or any combination of these learning methods. That is, any machine learning method may be used.

[0097] Based on the trained model 4003 and equipment information selected by a user and displayed in the second display region 320, the processing unit 43 searches a three-dimensional point cloud for a point cloud corresponding to equipment indicated by the equipment information, and acquires the position of the retrieved point cloud (i.e., subset point cloud) as a candidate for a two-dimensional position. The position of the subset point cloud is, for example, a position of the centroid of the subset point cloud.

[0098] The creation unit 47 creates the display screen 300 that displays an indication image to be superimposed over the three-dimensional point cloud in the first display region 310. The indication image indicates the candidate for the two-dimensional position acquired by the processing unit 43.

[0099] FIG. 17 is a diagram illustrating another example of the display screen 300 created by the creation unit 47. Differences between FIGs. 17 and 8 are that the first display region 310 displays a candidate pointer 312 and the second display region 320 additionally includes aFN202502325confirm candidate position button 324. The candidate pointer 312 is an example of the indication image indicating a candidate for the two-dimensional position.

[0100] The user checks the position of the candidate pointer 312 and presses the confirm candidate position button 324 to confirm the position of the candidate pointer 312 as the two-dimensional position of the equipment displayed in the second display region 320. In the present embodiment, the two-dimensional position of the equipment is confirmed based on the trained model 4003. This can facilitate association between the position information and the equipment information.

[0101] The user uses the accepting unit 12 to move the candidate pointer 312 and press the designate position button 321. In this manner, the user can designate the position of the candidate pointer 312 after the movement as the two-dimensional position. This can facilitate correction of the two-dimensional position determined based on the trained model 4003.

[0102] Instead of displaying the confirm candidate position button 324 in the second display region 320, the processing unit 43 may associate the position information and the equipment information with each other using the two-dimensional position determined based on the trained model 4003 without any modification. In this case, the user can move the position of the candidate pointer 312 and press the designate position button 321 to designate the position of the candidate pointer 312 after the movement as the two-dimensional position.

[0103] The training data for the trained model 4003 may further include three-dimensional information. Training using the three-dimensional information in addition to the two-dimensional image can increase the accuracy of the image search.

[0104] In the present embodiment, the position icon 322 illustrated in FIG. 9 may be displayed in the second display region 320. In this case, an image of the position icon 322 indicating the association based on the two-dimensional position designated by the user may be different from an image of the position icon 322 indicating the association based on the two-dimensional position that is determined based on the trained model 4003. For example, these images may be distinctive images different in shapes or colors. This allows the user to easily distinguish the equipment information associated with the position information based on the trained model 4003 and check or correct the position information when desired.

[0105] The position icon 322 indicating the association based on the two-dimensional position designated by the user is an example of a first distinctive image. The position icon 322 indicating the association based on the two-dimensional position determined based on the trained model 4003 is an example of a second distinctive image. Instead of making the images of the position icon 322 different, for example, a tag such as "Al detection result" may beFN202502325attached when the equipment information associated with the position information based on the trained model 4003 is displayed in the second display region 320.

[0106] FIG. 18 is a flowchart illustrating an example of a procedure of the association process according to the second embodiment. Since processing in steps S200 to S202 and S207 in FIG. 18 is substantially the same as the processing in steps S100 to S102 and S106 in FIG. 11, the description thereof is omitted.

[0107] In step S203, the processing unit 43 uses the trained model 4003 to acquire a candidate for the two-dimensional position. In step S204, the creation unit 47 creates the display screen 300 that displays an indication image indicating the candidate for the two-dimensional position in the first display region 310.

[0108] In step S205, the processing unit 43 determines whether the acquired candidate for the two-dimensional position is confirmed as the two-dimensional position of the equipment. When the acquired candidate for the two-dimensional position is confirmed as the two-dimensional position of the equipment by the user (step S205: Yes), the process proceeds to step S207. When the acquired candidate for the two-dimensional position is not confirmed as the two-dimensional position of the equipment by the user (step S205: No), the processing unit 43 accepts designation of the two-dimensional position of the equipment in step S206. More specifically, the user changes (moves) the position of the candidate for the two-dimensional position, and designates the two-dimensional position of the equipment. The processing unit 43 accepts the two-dimensional position of the equipment thus designated.

[0109] In the present embodiment, the two-dimensional position of the equipment is determined based on the trained model 4003. This can save the user the effort of designating the two-dimensional position.

[0110] The program to be executed by the information processing apparatus according to each of the embodiments described above is recorded on and provided through a non-transitory computer-readable recording medium such as a compact disc read-only memory (CD-ROM), a flexible disk (FD), a compact disc recordable (CD-R), or a digital versatile disc (DVD), in an installable or executable file format.

[0111] The program to be executed by the information processing apparatus according to each of the embodiments may be stored on a computer connected to a network such as the Internet, and downloaded and thus provided via the network. The program to be executed by the information processing apparatus according to each of the embodiments may be provided or distributed via a network such as the Internet.

[0112] FN202502325The program according to each of the embodiments may be preinstalled and provided on the ROM or any other memory.

[0113] The program to be executed by the information processing apparatus according to each of the embodiments has a module configuration including the above-described functional units (such as the processing unit 43, the creation unit 47, and the storing and reading unit 49). The CPU (i.e., a processor) that is actual hardware reads the program from the recording medium and executes the program, so that the above-described functional units are loaded to and generated in a main storage device.

[0114] The present invention can be implemented in any convenient form, for example using dedicated hardware, or a mixture of dedicated hardware and software. The present invention may be implemented as computer software implemented by one or more networked processing apparatuses. The processing apparatuses include any suitably programmed apparatuses such as a general purpose computer, a personal digital assistant, a Wireless Application Protocol (WAP) or third-generation (3G)-compliant mobile telephone, and so on. Since the present invention can be implemented as software, each and every aspect of the present invention thus encompasses computer software implementable on a programmable device. The computer software can be provided to the programmable device using any conventional carrier medium (carrier means). The carrier medium includes a transient carrier medium such as an electrical, optical, microwave, acoustic or radio frequency signal carrying the computer code. An example of such a transient medium is a Transmission Control Protocol / Internet Protocol (TCP / IP) signal carrying computer code over an IP network, such as the Internet. The carrier medium may also include a storage medium for storing processor readable code such as a floppy disk, a hard disk, a compact disc read-only memory (CD-ROM), a magnetic tape device, or a solid state memory device.

[0115] The functionality of the elements disclosed herein may be implemented using circuitry or processing circuitry which includes general purpose processors, special purpose processors, integrated circuits, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), and / or combinations thereof which are configured or programmed, using one or more programs stored in one or more memories, to perform the disclosed functionality. Processors are considered processing circuitry or circuitry as they include transistors and other circuitry therein. In the disclosure, the circuitry, units, or means are hardware that carry out or are programmed to perform the recited functionality. The hardware may be any hardware disclosed herein which is programmed or configured to carry out the recited functionality.There is a memory that stores a computer program which includes computer instructions. These computer instructions provide the logic and routines that enable the hardware (e.g., processing circuitry or circuitry) to perform the method disclosed herein. This computerFN202502325program can be implemented in known formats as a computer-readable storage medium, a computer program product, a memory device, a record medium such as a CD-ROM or DVD, and / or the memory of an FPGA or ASIC.

[0116] The apparatuses or devices described in the embodiments are merely one example of multiple computing environments that implement the embodiments disclosed herein.

[0117] In some embodiments, the management server 40 includes multiple computing devices, such as a server cluster. The multiple computing devices communicate with one another through any type of communication link including a network, a shared memory, or the like and perform the processes disclosed herein. Likewise, the terminal apparatus 10 may include multiple computing devices that communicate with one another.

[0118] The above-described embodiments are illustrative and do not limit the present invention. Thus, numerous additional modifications and variations are possible in light of the above teachings. For example, elements and / or features of different illustrative embodiments may be combined with each other and / or substituted for each other within the scope of the present invention. Any one of the above-described operations may be performed in various other ways, for example, in an order different from the one described above.

[0119] For example, aspects of the present disclosure include the following.According to Aspect 1, an information processing apparatus includes a creation unit and a processing unit. The creation unit creates a display screen including a first display region that displays a three-dimensional point cloud and a second display region that displays information on an article. The processing unit moves a subset point cloud of the three-dimensional point cloud relative to rest of the three-dimensional point cloud other than the subset point cloud. The creation unit creates the display screen displaying, in the first display region, the subset point cloud that has been moved by the processing unit and displaying, in the second display region, the information on the article associated with the subset point cloud before the movement of the subset point cloud in association with the subset point cloud after the movement.According to Aspect 2, in the information processing apparatus of Aspect 1, the subset point cloud before the movement includes at least one of a point cloud included in a bounding box obtained by object detection, a point cloud included in a segmented region obtained by region segmentation, and a point cloud included in a region designated by a user.According to Aspect 3, the information processing apparatus of Aspect 1 or 2, further includes a storage control unit. The storage control unit stores the information on the article associated with the subset point cloud before the movement and information on the subset point cloud after the movement in association with each other.FN202502325According to Aspect 4, in the information processing apparatus of Aspect 3, the information on the subset point cloud after the movement includes at least one of a point included in the subset point cloud, three-dimensional coordinates included in a region corresponding to the subset point cloud, a bounding box obtained by object detection, and a segmented region obtained by region segmentation.According to Aspect 5, in the information processing apparatus of Aspect 3 or 4, the storage control unit further stores the information on the article associated with the subset point cloud before the movement and three-dimensional coordinates of the subset point cloud before the movement in association with each other.According to Aspect 6, the information processing apparatus of any one of Aspects 1 to 5, further includes a display control unit and an accepting unit. The display control unit displays the display screen on a display. The accepting unit accepts an operation to move the subset point cloud relative to the rest of the three-dimensional point cloud displayed in the first display region. The processing unit is configured to move the subset point cloud relative to the rest of the three-dimensional point cloud, based on the operation accepted by the accepting unit.According to Aspect 7, an information processing method to be executed by an information processing apparatus, includes a creation step of creating a display screen including a first display region that displays a three-dimensional point cloud and a second display region that displays information on an article, and a processing step of moving a subset point cloud of the three-dimensional point cloud relative to rest of the three-dimensional point cloud. The creation step includes creating the display screen displaying, in the first display region, the subset point cloud that has been moved in the processing step and displaying, in the second display region, the information on the article associated with the subset point cloud before the movement of the subset point cloud in association with the subset point cloud after the movement.According to Aspect 8, a program causes a computer to function as creation means and processing means. The creation means creates a display screen including a first display region that displays a three-dimensional point cloud and a second display region that displays information on an article. The processing means moves a subset point cloud of the three-dimensional point cloud relative to rest of the three-dimensional point cloud. The creation means creates the display screen displaying, in the first display region, the subset point cloud that has been moved by the processing means and displaying, in the second display region, the information on the article associated with the subset point cloud before the movement of the subset point cloud in association with the subset point cloud after the movement.According to Aspect 9, an information processing system includes an information processing apparatus and a terminal apparatus to communicate with the information processing apparatus. The information processing system includes a creation unit, a display control unit, an accepting unit, and a processing unit. The creation unit creates a display screen including a first display region that displays a three-dimensional point cloud and a second display regionFN202502325that displays information on an article. The display control unit displays the display screen on a display. The accepting unit accepts an operation to move a subset point cloud of the three-dimensional point cloud displayed in the first display region relative to rest of the three-dimensional point cloud. The processing unit moves the subset point cloud relative to the rest of the three-dimensional point cloud, based on the operation accepted by the accepting unit. The creation unit creates the display screen displaying, in the first display region, the subset point cloud that has been moved by the processing unit and displaying, in the second display region, the information on the article associated with the subset point cloud before the movement of the subset point cloud in association with the subset point cloud after the movement.

[0120] This patent application is based on and claims priority to Japanese Patent Application No. 2025-028764, filed on February 26, 2025, in the Japan Patent Office, the entire disclosure of which is hereby incorporated by reference herein.[Reference Signs List]

[0121] 1 information processing system10 terminal apparatus12 accepting unit14 display control unit20 communication network40 management server43 processing unit47 creation unit49 storing and reading unit300 display screen310 first display region311 position pointer313 bounding box320 second display region321 designate position button322 position icon323 thumbnail image324 confirm candidate position button325 designate equipment button326 confirm movement button

Claims

FN202502325[CLAIMS]1. An information processing apparatus comprising:a creation unit configured to create a display screen including a first display region that displays a three-dimensional point cloud and a second display region that displays information on an article; anda processing unit configured to move a subset point cloud of the three-dimensional point cloud relative to rest of the three-dimensional point cloud other than the subset point cloud, whereinthe creation unit is configured to create the display screen displaying, in the first display region, the subset point cloud that has been moved and displaying, in the second display region, the information on the article associated with the subset point cloud before the movement of the subset point cloud in association with the subset point cloud after the movement.

2. The information processing apparatus according to claim 1, wherein the subset point cloud before the movement includes at least one of a point cloud included in a bounding box obtained by object detection, a point cloud included in a segmented region obtained by region segmentation, and a point cloud included in a region designated by a user.

3. The information processing apparatus according to claim 1 or 2, further comprising:a storage control unit configured to store the information on the article associated with the subset point cloud before the movement and information on the subset point cloud after the movement in association with each other.

4. The information processing apparatus according to claim 3, wherein the information on the subset point cloud after the movement includes at least one of a point included in the subset point cloud, three-dimensional coordinates included in a region corresponding to the subset point cloud, a bounding box obtained by object detection, and a segmented region obtained by region segmentation.

5. The information processing apparatus according to claim 3 or 4, wherein the storage control unit is configured to further store the information on the article associated with the subset point cloud before the movement and three-dimensional coordinates of the subset point cloud before the movement in association with each other.

6. The information processing apparatus according to any one of claims 1 to 5, further comprising:a display control unit configured to display the display screen on a display; andFN202502325an accepting unit configured to accept an operation to move the subset point cloud relative to the rest of the three-dimensional point cloud displayed in the first display region, whereinthe processing unit is configured to move the subset point cloud relative to the rest of the three-dimensional point cloud, based on the accepted operation.

7. An information processing system comprising:a creation unit configured to create a display screen including a first display region that displays a three-dimensional point cloud and a second display region that displays information on an article; anda display control unit configured to display the display screen on a display;an accepting unit configured to accept an operation to move a subset point cloud of the three-dimensional point cloud displayed in the first display region relative to rest of the three-dimensional point cloud other than the subset point cloud; anda processing unit configured to move the subset point cloud relative to the rest of the three-dimensional point cloud, based on the accepted operation, whereinthe creation unit is configured to create the display screen displaying, in the first display region, the subset point cloud that has been moved and displaying, in the second display region, the information on the article associated with the subset point cloud before the movement of the subset point cloud in association with the subset point cloud after the movement.

8. An information processing method performed by an information processing apparatus, comprising:creating a display screen including a first display region that displays a three-dimensional point cloud and a second display region that displays information on an article; andmoving a subset point cloud of the three-dimensional point cloud relative to rest of the three-dimensional point cloud other than the subset point cloud, whereinthe creating includes creating the display screen displaying, in the first display region, the subset point cloud that has been moved in the moving and displaying, in the second display region, the information on the article associated with the subset point cloud before the movement of the subset point cloud in association with the subset point cloud after the movement.

9. The information processing method according to claim 8, wherein the subset point cloud before the movement includes at least one of a point cloud included in a bounding box obtained by object detection, a point cloud included in a segmented region obtained by region segmentation, and a point cloud included in a region designated by a user.FN20250232510. The information processing method according to claim 8 or 9, further comprising:storing the information on the article associated with the subset point cloud before the movement and information on the subset point cloud after the movement in association with each other.

11. The information processing method according to claim 10, wherein the information on the subset point cloud after the movement includes at least one of a point included in the subset point cloud, three-dimensional coordinates included in a region corresponding to the subset point cloud, a bounding box obtained by object detection, and a segmented region obtained by region segmentation.

12. The information processing method according to claim 10 or 11, wherein the storing includes further storing the information on the article associated with the subset point cloud before the movement and three-dimensional coordinates of the subset point cloud before the movement in association with each other.

13. The information processing method according to any one of claims 8 to 12, further comprising:displaying the display screen on a display; andaccepting an operation to move the subset point cloud relative to the rest of the three-dimensional point cloud displayed in the first display region, whereinthe moving is based on the accepted operation.

14. Carrier means carrying computer readable code for controlling a computer system to carry out the method of any one of claims 8 to 13.

Images