Content processing system and content processing method

Abstract

To provide a content processing system and a content processing method, capable of streamlining a business activity so that a customer can concretely imagine a state after introducing a commercial material.SOLUTION: In a content processing system, a server device 20 comprises: a data storage unit 203 for AR service which stores a plurality of data where past site videos in which sites introduced with commercial materials in the past were captured are associated with composition data 29C of the commercial materials in site units; a site environment identification unit 202 which identifies a past site video most similar to the site video from among the past site videos where the site introduced with commercial materials in the past was captured on the basis of the site video transmitted from a client-side terminal 10; an AR processing unit 204 which creates AR content using composition data 29c of the commercial material associated with the past site videos; and an AR content transmission unit 205 which transmits the created AR content to the client-side terminal 10. The client-side terminal 10 comprises a video composition / output unit 105 which displays the AR content, superimposing it on the site video.SELECTED DRAWING: Figure 7

Description

Technical Field

[0001] The present invention relates to a content processing system and a content processing method for processing content related to merchandise proposed to customers.

Background Art

[0002] Conventionally, when a salesperson introduces merchandise to a customer during business negotiations, a specific image of the merchandise is conveyed to the customer by presenting explanatory materials or showing a demonstration video of the use of the merchandise on a PC or the like brought by the salesperson. The demonstration video visualizes the outline and features of the merchandise in an easy-to-understand manner so as to lead to the introduction of the merchandise.

[0003] For example, in Patent Document 1 below, as a technology for supporting the sales activities of a salesperson, a technology for proposing to the salesperson the customer's preferences and content constituting explanatory materials as a sales method is described.

[0004] In the technology described in Patent Document 1, a score is associated with the content, and the score is calculated so that the better the reputation of the content and the higher the number of uses, the higher the score. A predetermined number of contents with high scores are proposed to the salesperson. It is exemplified that the explanatory materials in Patent Document 1 are stored in the form of a PowerPoint file, a PDF file, or the like, and the content corresponds to the explanatory content in units of chapters of PowerPoint.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] However, there is a problem in that product descriptions and demonstration videos do not always make it easy for customers to concretely imagine what the situation will be like after the product is implemented. Furthermore, when there are multiple patterns for implementing the product, it is not easy to determine which of the multiple patterns is appropriate for the customer's environment.

[0007] If customers cannot concretely visualize the situation after implementing a product, sales representatives may have to expend considerable effort and time explaining it. Furthermore, there is a possibility that customers may give up on implementing the product because they cannot form a concrete image, resulting in the deal not being closed, or that customers may implement the product with a false image in mind, ultimately leading to distrust of the sales representative and the product provider.

[0008] This invention has been made in view of the above-mentioned problems, and aims to provide a content processing system and a content processing method that enable customers to concretely visualize the state after introducing a product, thereby improving the efficiency of sales activities. [Means for solving the problem]

[0009] To achieve the above objective, the content processing system according to the present invention is a content processing system in which a client-side terminal that can be deployed at the customer's site and a server device are connected via a network, The aforementioned client terminal, A video acquisition unit that acquires on-site video footage taken at the customer's site, A video transmission unit that transmits the aforementioned on-site video to the server device, A content receiving unit receives product display content transmitted from the server device, The system includes a video synthesis / output unit that combines the aforementioned on-site video footage with the aforementioned product display content to create a composite video in which the product footage is superimposed on the on-site video footage, and outputs it in a format that can be displayed on a display. The aforementioned server device A video receiving unit that receives the on-site video transmitted from the client terminal, A data storage unit that stores multiple data sets linked on a site-by-site basis, which include past site footage of sites where the product was previously installed and data for compositing the product. Based on the aforementioned on-site video, the on-site environment identification unit identifies the past on-site video that is most similar to the aforementioned on-site video from among past on-site video footage of sites where the product has been introduced in the past. A content creation unit creates product display content using product synthesis data associated with past on-site video footage identified by the aforementioned on-site environment identification unit, The system is characterized by comprising a content transmission unit that transmits the aforementioned product display content to the client-side terminal.

[0010] According to the content processing system described above, a demo environment for the product is placed on a server device accessible from the customer's environment. By recreating the virtual environment when the product is implemented in the demo environment, customers can virtually experience the product's implementation environment. In particular, by recreating the virtual environment using AR (Augmented Reality) or VR (Virtual Reality) technologies in the demo environment, customers can concretely visualize the product's implementation environment, and the effort and time required for sales representatives to explain the product can be reduced, thereby improving the efficiency of sales activities. Furthermore, by preparing multiple synthesis data 29c corresponding to multiple placement patterns for the same product, depending on the placement patterns of previously implemented products, it becomes possible to flexibly provide appropriate AR content tailored to the customer's on-site environment. Moreover, by placing the demo environment on a server device in the cloud, it becomes possible to recreate the virtual environment when the product is implemented in a simple system environment without requiring dedicated applications.

[0011] In the content processing system according to the present invention, the content creation unit embeds the improvement rate data associated with past on-site video footage identified by the on-site environment identification unit into the product display content. The video synthesis / output unit may create a synthesized video that displays the improvement rate data along with the video of the product.

[0012] According to the content processing system described above, the improvement rate of operational or production efficiency when introducing a product into a new customer environment can be displayed along with the product display content. This allows customers to visualize not only the state after introducing the product but also the effects of introducing it in a concrete way.

[0013] In the content processing system according to the present invention, the improvement rate data may be a numerical value that indicates the operational efficiency or production efficiency after the introduction of the product compared to the operational efficiency or production efficiency before the introduction of the product at a site where the product was introduced in the past.

[0014] According to the content processing system described above, it becomes possible to provide customers with specific numerical data on the improvement rate of operational or production efficiency when introducing products into a new customer environment.

[0015] In the content processing system according to the present invention, the content for displaying the product may be augmented reality content.

[0016] According to the content processing system described above, it becomes possible to overlay product display content created as AR content onto on-site video footage, particularly using AR technology.

[0017] The content processing system according to the present invention, in the above configuration, includes an operation reception unit that allows the client terminal to specify a product and outputs product specification information relating to the specified product, The on-site video identification unit of the server device may identify the past on-site video that is most similar to the on-site video from among the past on-site videos relating to the product specified in the product specification information.

[0018] According to the above content processing system, it becomes possible to accurately identify the merchandise that the customer is seeking and provide AR content related to that merchandise.

[0019] In the content processing system according to the present invention, in the above configuration, by referring to the past introduction performance data in which the merchandise was introduced in the past, a site unit data in which the past site video obtained by imaging the site where the merchandise was introduced in the past and the composite data of the merchandise are associated may be created by a data creation device.

[0020] According to the above content processing system, based on the introduction performance for each site where the merchandise was introduced in the past, it becomes possible to prepare composite data used for creating content for displaying the merchandise.

[0021] In the content processing system according to the present invention, in the above configuration, the data creation device calculates improvement rate data, which is a numerical value indicating the business efficiency or production efficiency after introducing the merchandise at a specific site with respect to the business efficiency or production efficiency before introducing the merchandise at the specific site, based on pre-introduction data, which is numerical data indicating the business efficiency or production efficiency before introducing the merchandise at a specific site, and post-introduction data, which is numerical data indicating the business efficiency or production efficiency after introducing the merchandise at the specific site.

[0022] According to the above content processing system, improvement rate data that can be displayed together with the merchandise is calculated based on the pre-introduction data and the post-introduction data, and specific and accurate improvement rate data related to the past merchandise introduction performance can be provided to the customer.

[0023] Further, in order to achieve the above object, a content processing method according to the present invention is a content processing method in a content processing system in which a client-side terminal that can be arranged at a customer site and a server device are connected via a network, a video acquisition step in which the client-side terminal acquires a site video taken at the customer site, The client terminal transmits the on-site video to the server device in a video transmission step, The server device performs a site environment identification step in which it refers to multiple data sets, each linked on a site-by-site basis, in which past site video footage of sites where the product has been previously installed and data for synthesizing the product, and identifies the past site video that is most similar to the site video in question, based on the site video. The server device performs a content creation step in which it creates content for displaying merchandise using synthesis data of merchandise associated with past site video identified in the site environment identification step, The server device transmits the product display content to the client terminal in a content transmission step, The client terminal combines the product display content received from the server device with the on-site video to create a composite video in which the product image is superimposed on the on-site video, and outputs it so that it can be displayed on a display; It is characterized by having the following features.

[0024] According to the content processing method described above, a demo environment for the product is placed on a server device accessible from the customer's environment, and the virtual environment when the product is implemented is reproduced in the demo environment, allowing customers to virtually experience the product's implementation environment. In particular, by reproducing the virtual environment using AR / VR technology in the demo environment, customers can concretely visualize the product's implementation environment, and the effort and time required for sales representatives to explain the product can be reduced, thereby improving the efficiency of sales activities. Furthermore, by placing the demo environment on a server device in the cloud, the virtual environment when the product is implemented can be reproduced in a simple system environment without requiring dedicated applications. [Effects of the Invention]

[0025] According to the content processing device and content processing method of the present invention, it is possible to improve the efficiency of sales activities by enabling customers to concretely visualize the state after introducing the product. [Brief explanation of the drawing]

[0026] [Figure 1] This figure shows an example of the configuration of a content processing system in an embodiment of the present invention. [Figure 2] This is a schematic block diagram showing an example of the hardware configuration of a client-side terminal in an embodiment of the present invention. [Figure 3] This is a schematic block diagram showing an example of the hardware configuration of a server device in an embodiment of the present invention. [Figure 4] This is a schematic block diagram showing an example of the hardware configuration of a database management system in an embodiment of the present invention. [Figure 5] This is a schematic block diagram showing an example of the hardware configuration of a data creation device for AR services in an embodiment of the present invention. [Figure 6] This is a functional block diagram showing an example of functions implemented in the database management system and AR service data creation device that constitute the content processing system in an embodiment of the present invention. [Figure 7] This is a functional block diagram showing an example of functions implemented in the client-side terminal and server device that constitute the content processing system in an embodiment of the present invention. [Figure 8] This flowchart shows an example of the data creation process for AR services performed in the content processing system according to an embodiment of the present invention. [Figure 9] This is a sequence diagram showing an example of AR content provisioning processing performed in the content processing system according to an embodiment of the present invention. [Figure 10] Figure 9 is a flowchart showing an example of the field information transmission process in step S20. [Figure 11] Figure 9 is a flowchart showing an example of the AR content transmission process in step S30. [Figure 12] Figure 9 is a flowchart showing an example of the AR display process in step S40. [Figure 13]In an embodiment of the present invention, the figure shows an example of a display screen on a head-mounted display connected to a client terminal, where (a) is a display screen showing on-site video captured by a camera, and (b) is a display screen showing a composite image in which AR content is superimposed on on-site video captured by a camera. [Modes for carrying out the invention]

[0027] Embodiments of the present invention will be described below with reference to the drawings.

[0028] <System Configuration> The configuration of the content processing system 1 in an embodiment of the present invention will be described with reference to Figure 1. Figure 1 is a diagram showing an example of the configuration of the content processing system 1 in an embodiment of the present invention. The content processing system according to the present invention is not limited to the configuration shown in Figure 1.

[0029] The content processing system 1 shown in Figure 1 is configured, as an example, to include a client-side terminal 10, a server device 20, a database management system 30, and an AR service data creation device 40.

[0030] The client-side terminal 10 is a terminal that can be operated by a sales representative or the customer themselves when conducting business negotiations with the customer as part of sales activities. The client-side terminal 10 is a portable terminal such as a portable notebook personal computer or tablet terminal, and may be owned and managed by the sales representative and usable when visiting the customer's location, or it may be owned and managed by the customer themselves. There are usually multiple client-side terminals 10, and two client-side terminals 10 are illustrated in Figure 1.

[0031] The client terminal 10 is connected to a network 50 such as the Internet and is configured to communicate with other devices, including a server device 20, which is also connected to the network 50, via the network 50. The client terminal 10 is configured to transmit on-site information, such as on-site video, to the server device 20 and to receive AR content from the server device 20.

[0032] This specification describes the use of AR technology, but VR technology may also be used. The AR content in this specification constitutes the product display content of the present invention.

[0033] The server device 20 is a server device that implements sales support functions to assist the sales activities of sales representatives. The server device 20 is connected to a network 50 such as the Internet, and is capable of communicating with other devices, including client-side terminals 10, which are also connected to the network 50, via the network 50.

[0034] The server device 20 functions as a web server, for example, located on the cloud, and also functions as an AR server that provides AR services, and is configured to provide AR content to the client terminal 10. Preferably, the server device 20 has an authentication / authorization function and grants access rights only to specific client terminals 10, and preferably, communication between the server device 20 and the specific client terminal 10 is kept confidential. The server device 20 may consist of a single device, or it may be a distributed server configuration in which the functions are distributed among multiple devices.

[0035] The database management system 30 is a device that manages information (customer data) about multiple customer companies. The database management system 30 has authentication / authorization functions and is configured to be accessible only from specific devices. In this embodiment, the database management system 30 accepts access from the AR service data creation device 40 and can provide the requested data to the AR service data creation device 40.

[0036] The AR service data creation device 40 is a device that creates AR service data (source data for AR content referenced by the server device 20) to be provided to the client terminal 10 by the server device 20 by referencing and utilizing data managed by the database management system 30. In this embodiment, the AR service data creation device 40 can access the database management system 30 and obtain the data necessary for creating AR service data from the database management system 30. Furthermore, the AR service data creation device 40 can store the AR service data in a location accessible by the server device 20 (for example, within the storage 28 of the server device 20 shown in Figure 3).

[0037] The database management system 30 and the AR service data creation device 40 are located in the backend, invisible to the client terminal 10. Figure 1 illustrates a configuration in which communication between the database management system 30 and the AR service data creation device 40, and communication between the AR service data creation device 40 and the server device 20, is performed via network 50 using a VPN (Virtual Private Network). However, this may also be done via a dedicated network such as a company LAN (Local Area Network).

[0038] <Configuration of client-side terminal 10> The configuration of the client-side terminal 10 in an embodiment of the present invention will be described with reference to Figure 2. Figure 2 is a schematic block diagram showing an example of the hardware configuration of the client-side terminal 10 in an embodiment of the present invention. The client-side terminal according to the present invention is not limited to the configuration shown in Figure 2.

[0039] As described above, the client-side terminal 10 can be implemented using a portable terminal. As an example, as shown in Figure 2, the client-side terminal 10 has a processor 11, memory 12, communication unit 13, operation input unit 14, monitor 15, microphone 16, camera 17, and storage 18, and each component is connected via a bus 10a.

[0040] The processor 11 is hardware that executes various instructions written in a program and realizes and controls various functions on the client terminal 10. The processor 11 may be, for example, a CPU (Central Processing Unit), a DSP (Digital Signal Processor) or a GPU (Graphics Processing Unit) that performs data processing specialized for a specific purpose.

[0041] Memory 12 is volatile memory that temporarily stores programs executed on the client terminal 10 and data to be processed. Memory 12 is a main memory such as RAM (Random Access Memory).

[0042] The communication unit 13 has a function that allows the client terminal 10 to communicate with other devices. In this embodiment, the client terminal 10 can communicate with the server device 20 via the network 50 using the communication unit 13.

[0043] The operation input unit 14 has the function of receiving information input from the user of the client terminal 10, and represents, for example, an input device such as a mouse and keyboard. The operation input unit 14 and the monitor 15 may be a touch panel type display in which their respective functions are integrated.

[0044] The microphone 16 has a sound pickup function, and the camera 17 has a photo capture function. The user of the client terminal 10 can use the microphone 16 and camera 17 to record and photograph the customer's on-site environment, business negotiations, etc. However, the client terminal 10 in this embodiment does not necessarily need to be equipped with the microphone 16. In addition, the client terminal 10 may be appropriately equipped with various other devices, such as a speaker.

[0045] The client terminal 10 is configured to display AR content received from the server device 20 on the monitor 15 by superimposing it onto the video captured by the camera 17. To improve the sense of presence and immersion regarding the video with the superimposed AR content, it is preferable that the monitor 15 and camera 17 are head-mounted displays. The head-mounted display used in this embodiment is preferably wearable on the user's head and equipped with an acceleration sensor and a gyroscope sensor to detect head movements (position, orientation, velocity, and acceleration, etc.), so that the display mode of the video with the superimposed AR content changes according to the user's movements.

[0046] Storage 18 is an auxiliary storage device that can be implemented using, for example, magnetic disks such as HDDs (Hard Disk Drives), semiconductor memory such as SSDs (Solid State Drives), magneto-optical disks, and optical disks.

[0047] The storage 18 is capable of storing programs and data in which desired processing procedures are described as program instructions. For example, as shown in Figure 2, the storage 18 stores various programs such as a web browser program 18a, a video acquisition program 18b, a video transmission program 18c, an AR content reception program 18d, a video synthesis / output program 18e, and an operation reception program 18f, as well as various data such as audio data 19a and video data 19b. The audio data 19a and video data 19b may be stored in a single file based on a specific digital multimedia container format.

[0048] The processor 11 reads each program from the storage 18 as appropriate, loads it into the memory 12, and executes program instructions, thereby enabling the client terminal 10 to realize the functions corresponding to each program. For example, the client terminal 10 can execute a web browser program 18a to display a web page received from the server device 20 which has web server functionality, and can send text data entered by the user using the operation input unit 14, video data 19b captured by the camera 17, etc., to the server device 20.

[0049] Storage 18 may store various programs and data other than those mentioned above. For example, storage 18 may store various programs such as the operating system program necessary for the operation of the client terminal 10, a recording processing program that processes the sound picked up by the microphone 16 into audio data 19a in file format, and a shooting processing program that processes the video captured by the camera 17 into video data 19b in file format, as well as various data such as communication setting information, communication history information, and negotiation result data entered by the user.

[0050] Preferably, the client terminal 10 can access the AR environment provided by the server device 20 simply by using, for example, a web browser, to use and display AR content. In this case, the client terminal 10 may be, for example, a general personal computer or tablet terminal equipped with a monitor 15 and a camera 17.

[0051] <Configuration of Server Device 20> The configuration of the server device 20 in an embodiment of the present invention will be described with reference to Figure 3. Figure 3 is a schematic block diagram showing an example of the hardware configuration of the server device 20 in an embodiment of the present invention. The server device according to the present invention is not limited to the configuration shown in Figure 3.

[0052] As described above, the server device 20 can be implemented by a server device located on the cloud. As an example, as shown in Figure 3, the server device 20 has a processor 21, memory 22, communication unit 23, operation input unit 24, monitor 25, and storage 28, and each component is connected via a bus 20a.

[0053] The processor 21 is hardware that executes various instructions written in a program and realizes and controls various functions in the server device 20. The processor 21 may be, for example, a CPU, a DSP or GPU that performs data processing specialized for a specific purpose, etc.

[0054] Memory 22 is volatile memory that temporarily stores programs executed by the server device 20 and data to be processed. Memory 22 is, for example, main memory such as RAM.

[0055] The communication unit 23 has a function that allows the server device 20 to communicate with other devices. In this embodiment, the server device 20 can communicate with the client terminal 10 via the network 50 using the communication unit 23.

[0056] The operation input unit 24 has the function of receiving information input from the user of the server device 20, and represents input devices such as a mouse and keyboard. The monitor 25 has the function of outputting visual information to the user and has a display screen. The operation input unit 24 and the monitor 25 may be a touch panel type display that integrates their respective functions.

[0057] Furthermore, the server device 20 may be appropriately equipped with various other devices, such as microphones, cameras, and speakers.

[0058] The storage 28 is an auxiliary storage device implemented by, for example, a magnetic disk such as an HDD, a semiconductor memory such as an SSD, a magneto-optical disk, or an optical disk. The storage 28 may be located in a network storage device separate from the server device 20, for example. In this embodiment, the storage 28 includes any data storage device accessible by the server device 20, such as storage on a network storage device.

[0059] The storage 28 is capable of storing programs and data in which desired processing procedures are described as program instructions. For example, as shown in Figure 3, the storage 28 stores various programs such as a web server program 28a, an AR service program 28b, a video reception program 28c, a field environment identification program 28d, an AR processing program 28e, an AR content transmission program 28f, and a specified information reception program 28g, as well as various data such as product identification information 29a, past field characteristic data 29b, synthesis data 29c, and improvement rate data 29d.

[0060] The processor 21 reads each program from the storage 28 as appropriate, loads it into the memory 22, and executes program instructions, thereby enabling the server device 20 to realize the functions corresponding to each program. In particular, the server device 20 can provide an AR environment (WebAR) to the client terminal 10 by executing a web server program 28a for functioning as a web server and an AR service program 28b for building a web-based AR service environment. Preferably, the client terminal 10 can use AR content and display AR content simply by accessing this AR environment using, for example, a web browser.

[0061] Storage 28 may store various programs and data other than those mentioned above. For example, storage 28 may store various programs such as operating system programs necessary for the operation of the server device 20.

[0062] <Configuration of Database Management System 30> The configuration of the database management system 30 in an embodiment of the present invention will be described with reference to Figure 4. Figure 4 is a schematic block diagram showing an example of the hardware configuration of the database management system 30 in an embodiment of the present invention. The database management system according to the present invention is not limited to the configuration shown in Figure 4.

[0063] As an example shown in Figure 4, the database management system 30 includes a processor 31, memory 32, communication unit 33, operation input unit 34, monitor 35, and storage 38, with each component connected via a bus 30a.

[0064] The processor 31 is hardware that executes various instructions written in a program and realizes and controls various functions in the database management system 30. The processor 31 may be, for example, a CPU, a DSP or GPU that performs data processing specialized for a particular purpose, etc.

[0065] Memory 32 is volatile memory that temporarily stores programs executed by the database management system 30 and data to be processed. Memory 32 is, for example, main memory such as RAM.

[0066] The communication unit 33 has a function for the database management system 30 to communicate with other devices. In this embodiment, the database management system 30 can communicate with the AR service data creation device 40 via the communication unit 33.

[0067] The operation input unit 34 has the function of receiving information input from the user of the database management system 30, and represents input devices such as a mouse and keyboard. The monitor 35 has the function of outputting visual information to the user and has a display screen. The operation input unit 34 and the monitor 35 may be a touch panel type display that integrates their respective functions.

[0068] The storage 38 is an auxiliary storage device implemented by, for example, a magnetic disk such as an HDD, a semiconductor memory such as an SSD, a magneto-optical disk, or an optical disk. The storage 38 may be located in a separate network storage device managed by, for example, the database management system 30. In this embodiment, the storage 38 includes any data storage device that can be managed by the database management system 30, such as storage on a network storage device.

[0069] The storage 38 is capable of storing programs and data in which desired processing procedures are described as program instructions. For example, as shown in Figure 4, the storage 38 stores various programs such as the database management program 38a.

[0070] The processor 31 reads each program from the storage 38 as needed, loads it into memory 32, and executes program instructions, thereby enabling the database management system 30 to realize the functions corresponding to each program. In particular, the database management system 30 can manage various types of data by relating them to each other, by executing a database management program 38a that functions as a database.

[0071] Storage 38 may store various programs and data other than those mentioned above. For example, storage 38 may store various programs, such as operating system programs necessary for the operation of the database management system 30.

[0072] The database management system 30 has a customer database 39a that stores customer data collected and accumulated by various means. Examples of data stored in the customer database 39a include customer information 39a1, past on-site video data 39a2, pre-implementation data 39a3, and post-implementation data 39a4.

[0073] Customer information 39a1 includes various customer-related information such as the customer's identification number, name, industry, identification number of the product actually introduced (sold) to each customer, and the introduction date. Past field video data 39a2 is video data of the environment of the customer where the product was introduced. Pre-introduction data 39a3 is numerical data showing the operational efficiency before the product was actually introduced. Post-introduction data 39a4 is numerical data showing the operational efficiency after the product was actually introduced. Operational efficiency includes, for example, product sales and profits, and production efficiency (product yield rate) on the factory production line.

[0074] Note that in this specification, "past" means a customer who introduced the specific merchandise before (in the past) when the customer to whom the specific merchandise is to be sold (the customer who is trying to introduce the specific merchandise) is regarded as the current customer. Therefore, "past site" in this specification means the site of a customer who introduced a specific merchandise in the past, and specifically means the site where the specific merchandise was actually introduced.

[0075] The database management system 30 has a merchandise database 39b that stores data related to merchandise that can be provided to customers. Examples of the data stored in the merchandise database 39b include merchandise information 39b1, merchandise graphics data 39b2 for each merchandise, and the like.

[0076] The data stored in the database management system 30 is stored and managed in association with each other. For example, based on the name of a customer, it is possible to identify the merchandise that has actually been introduced to that customer, past site video data 39a2 that captures the customer's environment, pre-introduction data 39a3 that indicates the business efficiency before merchandise introduction, and post-introduction data 39a4. Also, for example, based on the name of a merchandise, it is possible to identify the customer to whom the merchandise has actually been introduced, past site video data 39a2 that captures the customer's environment, pre-introduction data 39a3 that indicates the business efficiency before merchandise introduction, and post-introduction data 39a4.

[0077] <Configuration of the AR service data creation device 40> Referring to FIG. 5, the configuration of the AR service data creation device 40 in an embodiment of the present invention will be described. FIG. 5 is a schematic block diagram showing an example of the hardware configuration of the AR service data creation device 40 in an embodiment of the present invention. The AR service data creation device according to the present invention is not limited to the configuration shown in FIG. 4.

[0078] As an example shown in Figure 5, the AR service data creation device 40 includes a processor 41, memory 42, communication unit 43, operation input unit 44, monitor 45, and storage 48, with each component connected via a bus 40a. The AR service data creation device 40 constitutes the data creation device of the present invention.

[0079] The processor 41 is hardware that executes various instructions written in a program and realizes and controls various functions in the AR service data creation device 40. The processor 41 may be, for example, a CPU, a DSP or GPU that performs data processing specialized for a particular purpose, etc.

[0080] Memory 42 is a volatile memory that temporarily stores programs executed by the AR service data creation device 40 and data to be processed. Memory 42 is, for example, a main memory such as RAM.

[0081] The communication unit 43 has a function that allows the AR service data creation device 40 to communicate with other devices. In this embodiment, the AR service data creation device 40 can communicate with the server device 20 and the database management system 30 via the communication unit 43.

[0082] The operation input unit 44 has the function of receiving information input from the user of the AR service data creation device 40, and represents input devices such as a mouse and keyboard. The monitor 45 has the function of outputting visual information to the user and has a display screen. The operation input unit 44 and the monitor 45 may be a touch panel type display that integrates their respective functions.

[0083] The storage 48 is an auxiliary storage device implemented by, for example, a magnetic disk such as an HDD, a semiconductor memory such as an SSD, a magneto-optical disk, or an optical disk. The storage 48 may be located in a separate network storage device managed by, for example, the AR service data creation device 40. In this embodiment, the storage 48 includes any data storage device that can be managed by the AR service data creation device 40, such as storage on such a network storage device.

[0084] The storage 48 is capable of storing programs and data in which desired processing procedures are described as program instructions. For example, as shown in Figure 5, the storage 48 stores various programs such as a past field feature data creation program 48a, a synthesis data creation program 48b, and an improvement rate data creation program 48c.

[0085] The processor 41 reads each program from the storage 48 as appropriate, loads it into the memory 42, and executes program instructions, thereby enabling the AR service data creation device 40 to realize the functions corresponding to each program.

[0086] Storage 48 may store various programs and data other than those mentioned above. For example, storage 48 may store various programs such as operation system programs necessary for the operation of the AR service data creation device 40, and various data such as AR service data created by the AR service data creation device 40.

[0087] <Functions of the database management system 30 and the data creation device 40 for AR services> Figure 6 is a functional block diagram showing an example of functions implemented in the database management system 30 and the AR service data creation device 40 that constitute the content processing system 1 in an embodiment of the present invention.

[0088] As shown in FIG. 6, the database management system 30 and the AR service data creation device 40 are configured to be able to communicate with each other via their respective communication units 33 and 43. Further, the AR service data creation device 40 is configured to create AR service data and to be able to store the AR service data in the storage 28 of the server device 20 through the communication unit 33 and the communication unit 23 of the server device 20.

[0089] <Functions of the database management system 30> As shown in FIG. 6, the database management system 30 has a database management unit 301.

[0090] The database management unit 301 has a function of managing the databases stored in the database management system 30. In the present embodiment, the database management unit 301 manages the customer database 39a and the merchandise database 39b, reads out the data specified by a request from the AR service data creation device 40, and can provide the read-out data to the AR service data creation device 40 through the communication unit 33. The database management unit 301 is realized by the processor 31 appropriately reading out the database management program 38a from the storage 38 and expanding it on the memory 32 and executing the program instructions.

[0091] <Functions of the AR service data creation device 40> As shown in FIG. 6, the AR service data creation device 40 has a past site feature data creation unit 401, a synthesis data creation unit 402, and an improvement rate data creation unit 403.

[0092] The past site feature data creation unit 401 has the function of creating past site feature data 29b based on data contained in the customer database 39a and product database 39b managed by the database management system 30, and the function of storing the past site feature data 29b in the storage 28 of the server device 20. The past site feature data creation unit 401 is realized when the processor 41 appropriately reads the past site feature data creation program 48a from the storage 48, loads it into memory 42, and executes program instructions.

[0093] Past site feature data 29b is data that shows the characteristics of customer sites where the product was actually implemented. Past site feature data 29b constitutes data for AR services.

[0094] As an example, the past site feature data creation unit 401 may read past site video data 39a2, which is video data of customer sites where a particular product has been introduced, from the database management system 30, and output video or still images extracted from the past site video data 39a2 as past site feature data 29b. In this case, the past site feature data 29b will be video or still images extracted from the past site video data 39a2. If the past site feature data 29b is data that includes extracted video or still images, it is preferable that the site environment identification unit 202 of the server device 20 is equipped with a video comparison function that compares multiple video or still images and determines their similarity, as will be described later.

[0095] As another example, the past site feature data creation unit 401 is equipped with a video analysis function that extracts feature points contained in the video. This video analysis function can utilize existing technologies in video data processing. The past site feature data creation unit 401 reads past site video data 39a2, which is video data of customer sites where a specific product has been introduced, from the database management system 30, analyzes the past site video data 39a2, and is capable of extracting feature points of various objects captured in it. In this case, the past site feature data 29b is data that includes feature points of various objects captured in the past site video data 39a2. When the past site feature data 29b is data that includes feature points of objects captured, it is preferable that the site environment identification unit 202 of the server device 20 is equipped with a video analysis function that extracts feature points contained in the video, as will be described later.

[0096] The past site feature data creation unit 401 is configured to create past site feature data 29b for each past site video data 39a2 (i.e., for each site). Therefore, even for the same product, if the customer to whom the product is introduced is different, and consequently the past site video data 39a2 is different, multiple past site feature data 29b will be created. The past site feature data 29b is stored in the storage 28 of the server device 20, associated with product identification information 29a that identifies the product.

[0097] The data creation unit 402 has the function of creating data for synthesis 29c based on data contained in the customer database 39a and product database 39b managed by the database management system 30, and the function of storing the data for synthesis 29c in the storage 28 of the server device 20. The data creation unit 402 is realized when the processor 41 reads the data for synthesis 48b from the storage 48 as appropriate, loads it into memory 42, and executes program instructions.

[0098] The data for synthesis 29c is data containing AR content provided from the server device 20 to the client terminal 10. The data for synthesis 29c constitutes the data for the AR service.

[0099] The synthesis data creation unit 402 is equipped with an AR content creation function that creates synthesis data 29c, such as 3DCG (3D Computer Graphics) data, that can be used as AR content, from the product graphics data 39b2. This AR content creation function can apply existing technologies in AR content creation processing. The synthesis data creation unit 402 reads the product graphics data 39b2 from the database management system 30, processes it, and is able to create synthesis data 29c that can be used as AR content in an AR environment (e.g., a WebAR environment) realized by the server device 20. The data format of the synthesis data 29c that can be used as AR content is not particularly limited, but for example, 3DCG data can be used.

[0100] The data creation unit 402 for synthesis is configured to create synthesis data 29c for each past on-site video data 39a2 (i.e., for each on-site). Therefore, even if the same product is introduced to different on-site locations, each on-site location will have a different set, and multiple patterns of synthesis data 29c will be created for the same product. For example, there may be differences in the orientation of wiring for a single product (equipment placed on a production line, etc.), differences in vertical / horizontal placement, and differences in the placement position of each part if a single product is composed of multiple parts. Synthesis data 29c for each state (pattern) is created according to these differences. The synthesis data 29c is stored in the storage 28 of the server device 20, associated with product identification information 29a that identifies the product.

[0101] The improvement rate data creation unit 403 has the function of creating improvement rate data 29d based on data contained in the customer database 39a and product database 39b managed by the database management system 30, and the function of storing the improvement rate data 29d in the storage 28 of the server device 20. The improvement rate data creation unit 403 is realized when the processor 41 reads the improvement rate data creation program 48c from the storage 48 as appropriate, loads it into memory 42, and executes program instructions.

[0102] The improvement rate data 29d is data that quantifies the improvement rate of business efficiency before and after the introduction of the product. The improvement rate data 29d constitutes the data for the AR service.

[0103] The improvement rate data creation unit 403 reads from the database management system 30 pre-implementation data 39a3, which is numerical data showing the operational efficiency before the actual implementation of a particular product, and post-implementation data 39a4, which is numerical data showing the operational efficiency after the actual implementation of the product, for a specific customer who has introduced a particular product. The improvement rate data creation unit 403 is capable of calculating improvement rate data 29d from the pre-implementation data 39a3 and post-implementation data 39a4. For example, the improvement rate data creation unit 403 may calculate a value by dividing the post-implementation data 39a4 by the pre-implementation data 39a3 and multiplying by 100 (i.e., 100 × post-implementation data / pre-implementation data (%)) and use the result of this calculation as the improvement rate data 29d.

[0104] The improvement rate data creation unit 403 is configured to create improvement rate data 29d for each product actually introduced to a customer. The improvement rate data 29d is stored in the storage 28 of the server device 20, associated with product identification information 29a that identifies the product and past field characteristic data 29b.

[0105] The AR service data creation device 40 may be composed of multiple data processing devices. For example, a data processing device equipped with a past site feature data creation unit 401, a data processing device equipped with a synthesis data creation unit 402, and a data processing device equipped with an improvement rate data creation unit 403 may each be provided as separate data processing devices.

[0106] <Functions of client-side terminal 10 and server device 20> Figure 7 is a functional block diagram showing an example of functions implemented in the client-side terminal 10 and server device 20 that constitute the content processing system 1 in an embodiment of the present invention.

[0107] As shown in Figure 7, the client terminal 10 and the server device 20 are configured such that their respective communication units 13 and 23 are connected to the network 50, and they can communicate with each other via the network 50.

[0108] <Functions of client-side terminal 10> The client-side terminal 10 includes a display (HMD) 101, a video acquisition unit (camera) 102, a video transmission unit 103, an AR content receiving unit 104, a video synthesis / output unit 105, and an operation reception unit 106.

[0109] The display 101 is a display device that allows customers to view various information, including AR content. The display 101 corresponds to the monitor 15 shown in Figure 2. As mentioned above, it is preferable to use a head-mounted display equipped with a camera 17 in order to improve the user's sense of presence and immersion.

[0110] The video acquisition unit 102 has the function of capturing video of the customer site using the camera 17. The video acquisition unit 102 is realized when the processor 11 reads the video acquisition program 18b from the storage 18 as appropriate, loads it onto the memory 12, and executes program instructions.

[0111] The video transmission unit 103 has the function of transmitting on-site video data (on-site video) captured by the video acquisition unit 102 to the server device 20. The on-site video data transmitted from the video transmission unit 103 to the server device 20 is used as reference data when the server device 20 selects the composite data 29c. The video transmission unit 103 is realized when the processor 11 reads the video transmission program 18c from the storage 18 as appropriate, loads it onto the memory 12, and executes program instructions.

[0112] The AR content receiving unit 104 has the function of receiving AR content transmitted from the server device 20. The AR content receiving unit 104 is realized when the processor 11 reads the AR content receiving program 18d from the storage 18 as appropriate, loads it onto the memory 12, and executes program instructions.

[0113] The video synthesis / output unit 105 has the function of synthesizing AR content received from the server device 20 and video acquired by the video acquisition unit 102, outputting the result, and displaying it on the display 101. The video synthesis / output unit 105 is realized when the processor 11 reads the video synthesis / output program 18e from the storage 18 as appropriate, loads it onto the memory 12, and executes program instructions.

[0114] The video synthesis / output unit 105 does not require a special application and may use, for example, functions implemented in a general browser. This allows AR content to be superimposed on the video captured by the camera 17, similar to general WebAR that can be displayed in a browser. Furthermore, the video synthesis / output unit 105 can change the display mode of the AR content on the display 101 based on information regarding the operation of the AR content supplied from the operation reception unit 106.

[0115] The operation reception unit 106 has the function of receiving information such as the specification of the product and the synthesis data 29c, and operations to adjust the orientation and size of the superimposed AR content. Operations received by the operation reception unit 106 include not only input from the operation input unit 14 shown in Figure 2, but also head movements detected by the gyro sensor and accelerometer when the display 101 is a head-mounted display. The operation reception unit 106 is realized when the processor 11 reads the operation reception program 18f from the storage 18 as appropriate, loads it onto the memory 12, and executes program instructions.

[0116] If the operation received by the operation reception unit 106 relates to the operation of AR content, it is supplied to the video synthesis / output unit 105. Alternatively, for example, a list of products and synthesis data 29c may be displayed on a browser, allowing the customer or sales representative to specify products or synthesis data 29c. When the operation reception unit 106 receives specification information for products or synthesis data 29c selected from the list as described above, that specification information is sent to the server device 20.

[0117] <Server device 20 functions> The server device 20 includes a video / operation receiving unit 201, a field environment identification unit 202, an AR service data storage unit 203, an AR processing unit 204, an AR content transmission unit 205, and a designated information receiving unit 206.

[0118] The video / operation receiving unit 201 has the function of receiving specified information for merchandise and composite data 29c transmitted from the client terminal 10, or on-site video data captured by the client terminal 10. The video / operation receiving unit 201 is realized when the processor 21 appropriately reads the video / operation receiving program 28c from the storage 28, loads it onto the memory 22, and executes program instructions. The video / operation receiving unit 201 constitutes the video receiving unit of the present invention.

[0119] The field environment identification unit 202 has the function of identifying past field footage that is most similar to the field footage contained in the field footage data captured by the client terminal 10, and notifying the AR processing unit 204. The field environment identification unit 202 is realized when the processor 21 reads the field environment identification program 28d from the storage 28 as appropriate, loads it into the memory 22, and executes program instructions.

[0120] The field environment identification unit 202 includes, for example, a video comparison function that compares multiple video or still images to determine their similarity. This video comparison function can utilize existing technologies in video data processing.

[0121] If the past site feature data 29b included in the AR service data is data containing video or still images extracted from past site video data 39a2, the site environment identification unit 202 is configured to compare the site video received from the client terminal 10 with the video or still images included in the past site feature data 29b. The site environment identification unit 202 then identifies the past site feature data 29b containing video or still images with a high degree of match (similarity) with the site video, and notifies the AR processing unit 204 of the synthesis data 29c associated with this identified past site feature data 29b.

[0122] Furthermore, the on-site environment identification unit 202 also includes, as another example, a video analysis function that extracts feature points contained within the video. This video analysis function can utilize existing technologies in video data processing.

[0123] If the past site feature data 29b included in the AR service data contains feature points of various objects captured in the past site video data 39a2, the site environment identification unit 202 is configured to extract feature points of various objects captured in the site video received from the client terminal 10. The site environment identification unit 202 then identifies the past site feature data 29b containing feature points that have a high degree of agreement with the feature points of various objects captured in the site video, and notifies the AR processing unit 204 of the synthesis data 29c associated with this identified past site feature data 29b.

[0124] Furthermore, if the designated information receiving unit 206 receives designated product information from the client terminal 10, it is preferable that the field environment identification unit 202 extracts multiple past field feature data 29b associated with the specific product included in this designated information, and from among the extracted past field feature data 29b, it identifies past field feature data 29b that contain feature points with a high degree of agreement with the feature points of various objects captured in the field video, or past field feature data 29b that contain video or still images with a high degree of agreement (similarity) with the field video.

[0125] The AR service data storage unit 203 has the function of storing various information used in the creation of AR content. The AR service data storage unit 203 corresponds to the storage 28 in Figure 3, and stores the AR service data created by the AR service data creation device 40. The AR service data storage unit 203 constitutes the data storage unit of the present invention.

[0126] The AR service data is created based on the implementation results of each site where the product has been introduced in the past. It is a set of information consisting of product identification information 29a, past site characteristic data 29b, synthesis data 29c, and improvement rate data 29d, grouped by past site. The AR service data is created for each of multiple past sites, and the AR service data storage unit 203 stores multiple sets of AR service data. Note that even for the same product, if it has been introduced to different sites, the set will be different for each site, so multiple patterns of synthesis data 29c are prepared for the same product.

[0127] The AR processing unit 204 has the function of creating AR content for a product from the synthesis data 29c. The AR processing unit 204 is realized when the processor 21 reads the AR processing program 28e from the storage 28 as appropriate, loads it onto the memory 22, and executes program instructions. The AR processing unit 204 constitutes the content creation unit of the present invention.

[0128] The AR processing unit 204 reads the synthesis data 29c, which the field environment identification unit 202 has determined to be associated with past field feature data 29b, from the AR service data storage unit 203, and can create AR content, which is product display content, from this synthesis data 29c.

[0129] Furthermore, if the designated information receiving unit 206 receives designated information for the synthesis data 29c from the client terminal 10, the field environment identification unit 202 may read the synthesis data 29c included in this designated information from the AR service data storage unit 203 and create AR content, which is product display content, from this synthesis data 29c.

[0130] Furthermore, when the AR processing unit 204 creates AR content using the synthesis data 29c, it may read the improvement rate data 29d stored in the AR service data storage unit 203 in association with the synthesis data 29c, and embed the numerical values ​​contained in the improvement rate data 29d into the AR content so that they can be displayed.

[0131] The AR content transmission unit 205 has the function of transmitting AR content created by the AR processing unit 204 to the client terminal 10. The AR content transmission unit 205 is realized when the processor 21 appropriately reads the AR content transmission program 28f from the storage 28, loads it onto the memory 22, and executes program instructions. The AR content transmission unit 205 constitutes the content transmission unit of the present invention.

[0132] The designated information receiving unit 206 has the function of receiving designated information of products and synthesis data 29c transmitted from the client terminal 10. The client terminal 10 may be configured so that the user can specify products and synthesis data 29c as described above, and the designated information receiving unit 206 is configured to appropriately supply the products and synthesis data 29c included in the designated information to the field environment identification unit 202 or the AR processing unit 204.

[0133] <Processing performed by Content Processing System 1> Next, the processing performed by the content processing system 1 in the embodiment of the present invention will be described.

[0134] The processes performed by the content processing system 1 are broadly divided into two categories: data creation processes for AR services performed in the backend, and AR content provision processes that utilize AR content on the client-side terminal 10.

[0135] First, the process for creating data for AR services will be described. Figure 8 is a flowchart showing an example of the process for creating data for AR services performed in the content processing system 1 in an embodiment of the present invention. Note that each process shown in Figure 8 is mainly performed by the AR service data creation device 40.

[0136] In Figure 8, the AR service data creation device 40, in cooperation with the database management system 30, reads past site video data 39a2, which is video data of customer sites where a specific product has been introduced (step S11).

[0137] Next, the AR service data creation device 40 analyzes the past on-site video data 39a2 read in step S11 and extracts feature points of various objects captured in the video of the past on-site video data 39a2 as feature points of the video (step S12).

[0138] Next, the AR service data creation device 40 stores the feature points of the video extracted in step S12 as past site feature data 29b in the AR service data storage unit 203 of the server device 20 (step S13).

[0139] Furthermore, the past site feature data 29b can be any data that shows the characteristics of customer sites where the product was actually introduced, and is not limited to a method in which feature points of video extracted from the past site video data 39a2 are used as the past site feature data 29b. For example, as described above, video or still images extracted from the past site video data 39a2 may be stored as the past site feature data 29b in the AR service data storage unit 203 of the server device 20.

[0140] Next, the AR service data creation device 40, in cooperation with the database management system 30, reads out product graphics data 39b2 related to a specific product (step S14).

[0141] Next, the AR service data creation device 40 analyzes the arrangement patterns of merchandise shown in the video footage of the past site video data 39a2 read in step S11. Then, using the merchandise graphics data 39b2 related to a specific merchandise item, the AR service data creation device 40 creates composite data 29c that reproduces the state of merchandise placed at the past site as shown in the video footage of the past site video data 39a2 using AR content (step S15). In this case, the composite data 29c that reproduces the merchandise placed at the past site is created using the merchandise graphics data 39b2. However, for example, multiple composite data sets 29c could be prepared in advance, and a composite data set 29c that reproduces a state close to the arrangement at the past site could be selected from among these multiple composite data sets 29c.

[0142] Next, the AR service data creation device 40 stores the synthesis data 29c created in step S15 in the AR service data storage unit 203 of the server device 20 (step S16).

[0143] Next, the AR service data creation device 40, in cooperation with the database management system 30, reads pre-implementation data 39a3 and post-implementation data 39a4 related to the customer's site as seen in the past on-site video data 39a2 read in step S11 (step S17).

[0144] Next, the AR service data creation device 40 calculates improvement rate data 29d based on the pre-implementation data 39a3 and post-implementation data 39a4 read in step S11 (step S18). The improvement rate data 29d is, for example, the value obtained by dividing the post-implementation data 39a4 by the pre-implementation data 39a3 and multiplying by 100 (i.e., 100 × post-implementation data / pre-implementation data (%)).

[0145] Next, the AR service data creation device 40 stores the improvement rate data 29d calculated in step S18 in the AR service data storage unit 203 of the server device 20 (step S19).

[0146] Next, we will explain the AR content provision process.

[0147] Figure 9 is a sequence diagram showing an example of AR content provisioning processing performed by the content processing system 1 in an embodiment of the present invention.

[0148] In Figure 9, the client terminal 10 performs a field information transmission process to send customer site information as field information to the server device 20 (step S20). The server device 20 identifies appropriate synthesis data 29c based on the field information transmitted by the client terminal 10 and performs an AR content transmission process to send AR content using the identified synthesis data 29c to the client terminal 10 (step S30). The client terminal 10 performs an AR display process to overlay the AR content transmitted by the client terminal 10 onto the actual field video (step S40).

[0149] Figure 10 is a flowchart showing an example of the field information transmission process in step S20 of Figure 9. Each process shown in Figure 10 is executed by the client terminal 10.

[0150] In Figure 10, the client terminal 10 starts capturing video footage of the customer's site (on-site video) using the camera 17 (step S21). The on-site video data captured in step S21 is used by the server device 20 as a basis for making decisions to appropriately select the data 29c for synthesis.

[0151] The client terminal 10 acquires information specifying the products (products to be sold) to be proposed for introduction at the customer site (step S22). The product specification information includes identification information that specifically identifies the products. For example, the client terminal 10 may display a list of products, and the customer or sales representative may use the operation input unit 14 to select the desired product from the list of products. Note that the acquisition of product specification information in step S22 may be performed before the video recording process at the customer site in step S21.

[0152] Next, the client terminal 10 transmits the video footage of the customer site taken in step S21 and the specified product information specified in step S22 to the server device 20 (step S23).

[0153] Figure 11 is a flowchart showing an example of the AR content transmission process in step S30 of Figure 9. Each process shown in Figure 11 is executed by the server device 20.

[0154] In Figure 11, the server device 20 receives the on-site video and product specification information transmitted by the client terminal 10 in step S22 of Figure 10 (step S31).

[0155] The server device 20 refers to multiple AR service data stored in the AR service data storage unit 203 and identifies the past site feature data 29b that is most similar to the site video for a specific product specified in the product designation information (step S32).

[0156] As described above, the data for the AR service is a set of information consisting of product identification information 29a, past site characteristic data 29b, synthesis data 29c, and improvement rate data 29d, grouped by past site. The server device 20, for example, refers to the product identification information 29a to narrow down the AR service data corresponding to the product specified in the product specification information. At this time, if the data for the AR service corresponding to the product specified in the product specification information is narrowed down to only one, this AR service data is identified.

[0157] On the other hand, if there are multiple data sets for the AR service that have been narrowed down, the field video received in step S31 is matched with the past field feature data 29b to further narrow down the data for the AR service. If the past field feature data 29b is data that includes feature points from the past field video, the field environment identification unit 202 extracts the feature points contained in the field video received in step S31 and identifies the past field feature data 29b that contains the feature points with the highest degree of match with the field video. Also, if the past field feature data 29b is data that includes video or still images of the past field video, the field environment identification unit 202 performs a video comparison process between the field video received in step S31 and the past field video to identify the past field feature data 29b that contains the video or still images with the highest degree of match with the field video.

[0158] Next, the server device 20 identifies the synthesis data 29c associated with the past field feature data 29b identified in step S32 (step S33).

[0159] Then, the server device 20 reads the synthesis data 29c identified in step S33 and the improvement rate data 29d associated with the synthesis data 29c from the AR service data storage unit 203 (step S34).

[0160] The server device 20 creates AR content using the synthesis data 29c and improvement rate data 29d read in step S34 (step S35). The AR content includes AR material of the product and the improvement rate value included in the improvement rate data 29d, and is created so that the AR material of the product and the improvement rate can be displayed on the display 101 of the client terminal 10. Note that the AR content does not necessarily have to include the improvement rate value, and AR content that displays only the AR material of the product may be created.

[0161] The server device 20 transmits the AR content created in step S35 to the client terminal 10 (step S36).

[0162] Figure 12 is a flowchart showing an example of the AR display process in step S40 of Figure 9. Each process shown in Figure 12 is executed by the client terminal 10.

[0163] In Figure 12, the client terminal 10 receives the AR content transmitted by the server device 20 in step S36 of Figure 11 (step S41).

[0164] Next, the client terminal 10 combines the on-site video footage captured by the camera 17 with the AR content received in step S41 to create a display video (composite video) in which the AR content is superimposed on the on-site video footage (step S42).

[0165] Next, the client terminal 10 displays the video created in step S43 on the display 101 (step S43).

[0166] The processing in steps S42 and S43 described above continues until a termination condition is met, for example, when a termination instruction is entered by the user (step S44). As a result, the composite image in which the AR content is superimposed on the on-site video continues to be displayed on the display 101. Note that the client terminal 10 sends the on-site video to the server device 20 in step S23 so that the server device 20 can create the AR content, but when displaying the composite image in which the AR content is superimposed on the on-site video during the AR display processing, it is not necessary to send the on-site video to the server device 20.

[0167] In the AR display processing, the composite image, in which AR content is superimposed on on-site video footage, is preferably displayed using a head-mounted display equipped with a camera 17, and it is preferable that the customer considering introducing the product wears this head-mounted display. The head-mounted display is capable of displaying the composite image in accordance with the head movements of the wearer (in this case, the customer). By viewing the composite image, in which the AR content of the product is superimposed on on-site video footage captured by the camera 17, using the head-mounted display, the customer can virtually experience what it would be like when the product is actually introduced at their site (a virtual introduction environment).

[0168] Figure 13 shows an example of a display screen on a head-mounted display connected to a client terminal 10 in an embodiment of the present invention, where (a) shows a display screen showing on-site video captured by camera 17, and (b) shows a display screen showing a composite image in which AR content is superimposed on on-site video captured by camera 17.

[0169] The display screen shown in Figure 13(a) shows the on-site video footage captured by camera 17, that is, the on-site video footage without the AR content superimposed. On the other hand, the display screen shown in Figure 13(b) shows a composite image in which the AR content of the product is superimposed on the on-site video footage captured by camera 17 (the display state shown in Figure 13(a)).

[0170] The composite image includes product images made from AR materials of the product. This allows customers to visualize how the product would look when placed at their site by viewing the composite image displayed on the head-mounted display screen. Furthermore, by moving the head wearing the head-mounted display, or by operating the client terminal 10 and the head-mounted display, the orientation of the camera 17 (i.e., the direction in which the site video is captured), the product image superimposed on the site video, its position, size, etc., can be freely changed, allowing the display screen to be adjusted to show the desired state of the product placed at the customer's site.

[0171] Furthermore, this AR display uses composite data 29c that the server device 20 has identified from among multiple composite data 29c by matching it with video data from the customer's site. This makes it possible to display the product with the configuration suitable for the customer's site, even if, for example, there are multiple arrangement patterns for the product.

[0172] Furthermore, the numerical improvement rate included in the improvement rate data 29d may be displayed in the composite image. For example, as shown in Figure 13(b), displaying the improvement rate as a numerical value near the product image allows customers to concretely visualize the benefits of introducing the product.

[0173] As described above, the content processing system 1 in this embodiment allows customers to virtually experience the state in which the product is installed in their own on-site environment before the product is actually installed. This allows customers to concretely visualize the environment in which the product will be installed, reduces the effort and time required for sales representatives to explain the product, and improves the efficiency of sales activities. In addition, depending on the placement patterns of products that have been installed in the past, multiple synthesis data 29c corresponding to multiple placement patterns can be prepared for a specific product, and by matching them with video of the on-site environment, appropriate AR content tailored to the customer's on-site environment can be provided. Furthermore, by placing the demo environment on a server device 20 in the cloud, the virtual environment when the product is installed can be reproduced in a simple system environment without requiring a dedicated application or anything like that.

[0174] Furthermore, in the AR display, customers can see the degree of improvement in operational or production efficiency resulting from the actual implementation of the product, expressed as a specific numerical improvement rate. This allows them to visualize not only the state of the product after its implementation, but also the effects of its implementation in a concrete way.

[0175] The operation of the content processing system 1 in an embodiment of the present invention will be described.

[0176] In an embodiment of the present invention, the content processing system 1 is configured such that a client-side terminal 10 that can be deployed at the customer's site and a server device 20 are connected via a network 50.

[0177] The client terminal 10 includes a video acquisition unit 102 that acquires on-site video footage taken at the customer's site, a video transmission unit 103 that transmits the on-site video footage to the server device 20, an AR content receiving unit 104 that receives AR content transmitted from the server device 20, and a video synthesis / output unit 105 that combines AR content with the on-site video footage to create a composite video in which the video of the product is superimposed on the on-site video footage, and outputs it so that it can be displayed on the display 101.

[0178] Furthermore, the server device 20 includes a video / operation receiving unit 201 that receives on-site video transmitted from the client terminal 10, an AR service data storage unit 203 that stores multiple data sets associated on a site-by-site basis, each containing past on-site video footage of sites where products have been previously installed and product synthesis data 29c, a site environment identification unit 202 that identifies the past on-site video footage that is most similar to the current on-site video from among past on-site video footage of sites where products have been previously installed, based on the on-site video, an AR processing unit 204 that creates AR content using the product synthesis data 29c associated with the past on-site video identified by the site environment identification unit 202, and an AR content transmission unit 205 that transmits the AR content to the client terminal 10.

[0179] According to the above configuration, a demo environment for the product is placed on a server device 20 accessible from the customer's environment, and the virtual environment when the product is introduced is reproduced in the demo environment, allowing the customer to virtually experience the product's introduction environment. In particular, by reproducing the virtual environment using AR / VR technology in the demo environment, customers can concretely visualize the product's introduction environment, and the effort and time required for sales representatives to explain the product can be reduced, thereby improving the efficiency of sales activities. Furthermore, by preparing multiple synthesis data 29c corresponding to multiple placement patterns even for the same product, depending on the placement patterns of products that have been introduced in the past, it becomes possible to flexibly provide appropriate AR content tailored to the customer's on-site environment. Moreover, by placing the demo environment on a server device 20 in the cloud, the virtual environment when the product is introduced can be reproduced in a simple system environment without requiring dedicated applications.

[0180] In the content processing system 1 according to an embodiment of the present invention, the AR processing unit 204 may embed improvement rate data 29d associated with past on-site video footage identified by the on-site environment identification unit 202 into the AR content, and the video synthesis / output unit 105 may create a composite video that displays the improvement rate data 29d together with the video of the product.

[0181] According to the above configuration, the improvement rate of operational or production efficiency when introducing the product into a new customer environment can be displayed along with the product's AR content. This allows customers to visualize not only what the product looks like when it's implemented, but also the effects of implementing it.

[0182] In the content processing system 1 of the embodiment of the present invention, the improvement rate data 29d may be a numerical value that indicates the operational efficiency or production efficiency after the introduction of the product compared to the operational efficiency or production efficiency before the introduction of the product at a site where the product was introduced in the past.

[0183] According to the above configuration, it is possible to provide customers with specific numerical data on the improvement rate of operational or production efficiency when introducing the product into a new customer environment.

[0184] In the content processing system 1 according to an embodiment of the present invention, the client terminal 10 is equipped with an operation reception unit 106 that allows the user to specify a product and outputs product specification information relating to the specified product, and the on-site environment identification unit 202 of the server device 20 may identify the past on-site video that is most similar to the on-site video from among the past on-site videos relating to the product indicated in the product specification information.

[0185] With the above configuration, it is possible to accurately identify the products that customers are looking for and provide AR content related to those products.

[0186] The content processing system 1 in the embodiment of the present invention may include an AR service data creation device 40 that creates site-specific data by referencing past implementation data of products that have been introduced in the past, and associating past site video footage captured at sites where products have been introduced in the past with product synthesis data 29c.

[0187] According to the above configuration, synthesis data 29c used to create AR content can be prepared based on the implementation results of each site where the product has been introduced in the past.

[0188] In the content processing system 1 according to an embodiment of the present invention, the AR service data creation device 40 may calculate improvement rate data 29d, which is a numerical value indicating the operational efficiency or production efficiency after the introduction of a product relative to the operational efficiency or production efficiency before the introduction of a product at a specific site, based on pre-introduction data 39a3, which is numerical data indicating the operational efficiency or production efficiency before the introduction of a product at a specific site, and post-introduction data 39a4, which is numerical data indicating the operational efficiency or production efficiency after the introduction of a product at a specific site.

[0189] According to the above configuration, improvement rate data 29d, which can be displayed along with the product, can be calculated based on pre-implementation data 39a3 and post-implementation data 39a4, and specific and accurate improvement rate data 29d related to past product implementation results can be provided to the customer.

[0190] Furthermore, the content processing method in the embodiment of the present invention is a content processing method in a content processing system 1 in which a client-side terminal 10 that can be deployed at a customer site and a server device 20 are connected via a network 50. This content processing method includes a video acquisition step (step S11) in which the client-side terminal 10 acquires on-site video footage taken at a customer site, a video transmission step (step S13) in which the client-side terminal 10 transmits the on-site video footage to the server device 20, a site environment identification step (step S32) in which the server device 20 refers to a plurality of data in which past on-site video footage of sites where products have been previously introduced and product synthesis data 29c are associated on a site-by-site basis, and identifies the past on-site video footage that is most similar to the current on-site video footage from among the past on-site video footage of sites where products have been previously introduced, based on the current on-site video footage, and the server device 20 identifies in the site environment identification step The system includes an AR content creation step (step S35) in which AR content is created using composite data 29c of merchandise associated with past on-site video footage; a content transmission step (step S36) in which the server device 20 transmits the AR content to the client terminal 10; a video synthesis step (step S42) in which the client terminal 10 synthesizes the AR content received from the server device 20 onto the on-site video footage to create a composite video in which the video of the merchandise is superimposed on the on-site video footage; and a video output step (step S43) in which the client terminal 10 outputs the composite video created in the video synthesis step so that it can be displayed on the display 101.

[0191] According to the above process, a demo environment for the product is placed on a server device 20 accessible from the customer's environment, and the virtual environment when the product is implemented is reproduced in the demo environment, allowing the customer to virtually experience the product's implementation environment. In particular, by reproducing the virtual environment using AR / VR technology in the demo environment, customers can concretely visualize the product's implementation environment, and the effort and time required for sales representatives to explain the product can be reduced, thereby improving the efficiency of sales activities. Furthermore, by placing the demo environment on a server device 20 in the cloud, the virtual environment when the product is implemented can be reproduced in a simple system environment without requiring dedicated applications or the like.

[0192] The present invention is not limited to the embodiments described above, and its technical scope includes various modifications and design changes, etc., without departing from the technical spirit of the present invention. [Explanation of symbols]

[0193] 1. Content Processing System 10. Client-side terminal 10a, 20a, 30a, 40a bus 11, 21, 31, 41 processors 12, 22, 32, 42 memory 13, 23, 33, 43 Communications Department 14, 24, 34, 44 Operation Input Sections 15, 25, 35, 45 monitors 16 Mike 17 Cameras 18, 28, 38, 48 storage 18a Web browser program 18b Video Acquisition Program 18c Video transmission program 18d AR Content Reception Program 18e Video Compositing / Output Program 18f Operation Acceptance Program 19a Audio data 19b Video data 20 Server Devices 28a Web server program 28b AR Service Program 28c Video Reception Program 28d Site Environment Identification Program 28e AR Processing Program 28f AR Content Transmission Program 28g Specified Information Reception Program 29a Product identification information 29b Past Site Characteristics Data 29c Data for synthesis 29d Improvement Rate Data 30 Database Management Systems 38a Database Management Program 39a Customer Database 39a1 Customer information 39a2 Past on-site video data 39a3 Pre-implementation data 39a4 Post-implementation data 39b Product Database 39b1 Product Information 39b2 Product Graphics Data 40. Data creation device for AR services 48a Program for creating historical site characteristic data 48b Program for creating data for synthesis 48c Improvement Rate Data Creation Program 50 Networks 101 displays 102 Video Acquisition Unit 103 Video transmission unit 104 AR content receiving unit 105 Video Synthesis / Output Section 106 Operation Reception Section 201 Video / Control Receiver 202 Site Environment Identification Department 203 AR Service Data Storage Unit 204 AR Processing Unit 205 AR Content Transmission Unit 206 Designated Information Receiving Unit 301 Database Management Department 401 Past Site Characteristics Data Creation Department 402 Data Creation Section for Synthesis 403 Improvement Rate Data Creation Department

Claims

1. A content processing system in which a client-side terminal that can be deployed at the customer's site and a server device are connected via a network, The aforementioned client terminal, A video acquisition unit that acquires on-site video footage taken at the customer's site, A video transmission unit that transmits the aforementioned on-site video to the server device, A content receiving unit receives product display content transmitted from the server device, The system includes a video synthesis / output unit that combines the aforementioned on-site video footage with the aforementioned product display content to create a composite video in which the product footage is superimposed on the on-site video footage, and outputs it in a format that can be displayed on a display. The aforementioned server device A video receiving unit that receives the on-site video transmitted from the client terminal, A data storage unit that stores multiple data sets linked on a site-by-site basis, which include past site footage of sites where the product was previously installed and data for compositing the product. Based on the aforementioned on-site video footage, the on-site environment identification unit identifies past on-site video footage that is most similar to the on-site video footage in question. A content creation unit creates product display content using product synthesis data associated with past on-site video footage identified by the aforementioned on-site environment identification unit, A content processing system characterized by comprising a content transmission unit that transmits the aforementioned product display content to the client-side terminal.

2. The content creation unit embeds the improvement rate data associated with past on-site video footage identified by the on-site environment identification unit into the product display content. The content processing system according to claim 1, characterized in that the video synthesis / output unit creates a synthesized video that displays the improvement rate data together with the video of the product.

3. The content processing system according to claim 2, characterized in that the improvement rate data is a numerical value indicating the operational efficiency or production efficiency after the introduction of the product compared to the operational efficiency or production efficiency before the introduction of the product at a site where the product was introduced in the past.

4. The content processing system according to any one of claims 1 to 3, characterized in that the content for displaying the product is augmented reality content.

5. The aforementioned client-side terminal is equipped with an operation reception unit that allows the user to specify a product and outputs product specification information related to the specified product. The content processing system according to any one of claims 1 to 3, characterized in that the site environment identification unit of the server device identifies the past site video that is most similar to the site video from among past site videos relating to the product specified in the product designation information.

6. The content processing system according to any one of claims 1 to 3, characterized in that it has a data creation device that references past implementation data of products that have been introduced in the past, and creates site-specific data that associates past site video footage taken at sites where products have been introduced in the past with data for combining products.

7. The content processing system according to claim 6, characterized in that the data creation device calculates improvement rate data, which is a numerical value indicating the operational efficiency or production efficiency after the introduction of a product relative to the operational efficiency or production efficiency before the introduction of the product at a specific site, based on pre-introduction data, which is numerical data indicating the operational efficiency or production efficiency before the introduction of a product at a specific site, and post-introduction data, which is numerical data indicating the operational efficiency or production efficiency after the introduction of a product at the specific site.

8. A content processing method in a content processing system in which a client-side terminal that can be deployed at a customer site and a server device are connected via a network, The client terminal performs a video acquisition step in which it acquires on-site video footage taken at the customer's site, The client terminal transmits the on-site video to the server device in a video transmission step, The server device performs a site environment identification step in which it refers to multiple data sets, each linked on a site-by-site basis, in which past site video footage of sites where the product has been previously installed and data for synthesizing the product, and identifies the past site video that is most similar to the site video in question, based on the site video. The server device performs a content creation step in which it creates content for displaying merchandise using synthesis data of merchandise associated with past site video identified in the site environment identification step, The server device transmits the product display content to the client terminal in a content transmission step, The client terminal performs a video synthesis step in which it combines the product display content received from the server device with the on-site video to create a composite video in which the video of the product is superimposed on the on-site video. The client terminal outputs the composite video created in the video synthesis step so that it can be displayed on a display; A content processing method characterized by having the following:

Images