Program, information processing apparatus, method, and system

Abstract

To provide a technique that enables an operator to properly and easily perform necessary terminal operation.SOLUTION: There is provided a program that causes a computer including a memory and a processor to execute the steps of: accepting, for a predetermined function to be provided, resource data used for the function in response to a first action by an operator in one or more services; and specifying at least the function associated with a user in the services, and then presenting a candidate for a second action for executing the specified function to the operator.SELECTED DRAWING: Figure 11

Description

[Technical field]

[0001] The present invention relates to a program, an information processing device, a method, and a system. [Background technology]

[0002] In recent years, various technologies related to electronic commerce in which products are offered for sale via the Internet have been developed. For example, Patent Document 1 discloses an information processing device that automatically changes the orientation of a portrait-oriented product image to eliminate the inconvenience to the seller. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2011-180674 A Summary of the Invention [Problem to be solved by the invention]

[0004] As e-commerce services become more diverse, users need to create an account for each service and then use the functions provided by that service. In addition, as an operator, the user needs to install an app provided by each service, launch the app, and use the functions, which causes inconveniences such as time-consuming procedures.

[0005] An object of the present invention is to provide a technique that enables an operator to easily perform necessary terminal operations. [Means for solving the problem]

[0006] According to one aspect of the present disclosure, a program is provided that causes a computer having a memory and a processor to execute the steps of: accepting, through a first action of an operator, resource data to be used for a specific function provided in one or more services; identifying at least one of the functions associated with a user in the service; and presenting to the operator candidate second actions for executing the identified function.

[0007] According to the present disclosure, it is possible to easily carry out the tasks required for commercial transactions of goods via the Internet. [Brief description of the drawings]

[0008] [Figure 1] FIG. 2 is a diagram for explaining an overview of the processing of the system 1. [Diagram 2] 1 is a diagram illustrating a hardware configuration of a system 1 according to a first embodiment. [Diagram 3] 2 is a block diagram showing the functional configuration of the terminal device 10. FIG. [Figure 4] 2 is a block diagram showing the functional configuration of a management server 20. FIG. [Diagram 5] 2 is a block diagram showing a functional configuration of an external server 30. FIG. [Figure 6] 2 is a diagram showing a database included in a storage unit 202 of a management server 20. FIG. [Figure 7] FIG. 2 is a diagram showing a database provided in a storage unit 170 of the terminal device 10. [Figure 8] 10A to 10C are diagrams illustrating examples of screens displayed on the terminal device 10 when the first and second actions are accepted. [Figure 9] 1A to 1C are diagrams illustrating an example of how to use a selection dial 60 displayed on the terminal device 10. [Figure 10] 1 is a diagram illustrating an example of a management screen displayed on a terminal device 10. FIG. [Figure 11] 2 is a flowchart showing the flow of processing of the system 1 in the first embodiment. [Figure 12]FIG. 11 is a diagram illustrating the process flow of the data generation process (step S1110). [Figure 13] FIG. 11 is a diagram illustrating the process flow of the function execution process (step S1150). [Figure 14] FIG. 13 is a diagram illustrating an example of a screen in a modified example. [Figure 15] FIG. 11 is a diagram showing an example of a user database in a modified example. [Figure 16] FIG. 13 is a diagram showing a terminal device 10 in a modified example. [Figure 17] FIG. 1 is a diagram showing an overview of a commercial transaction management system 100 according to a second embodiment of the present invention. [Figure 18] 2 is a block diagram showing a configuration of a management server 20. FIG. [Figure 19] FIG. 2 is a diagram showing a database included in a storage unit 202. [Figure 20] FIG. 2 is a diagram illustrating an example of a screen of the terminal device 10. [Figure 21] FIG. 2 is a diagram for explaining the contents of operations managed by the commercial transaction management system 100. [Figure 22] FIG. 2 is a flow chart showing the process flow of the commercial transaction management system 100. [Figure 23] FIG. 13 is a block diagram showing a configuration of a management server 20 in a modified example. [Figure 24] FIG. 13 is a diagram illustrating example task candidates presented in a modified example. [Diagram 25] FIG. 11 is a diagram showing an overview of a commercial transaction management system 100 according to a third embodiment. [Figure 26] FIG. 11 is a diagram for explaining the contents of work managed by the commercial transaction management system 100 in the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. In the following description, the same components are denoted by the same reference numerals. The names and functions of the components are also the same. Therefore, detailed description thereof will not be repeated.

[0010] <1. First embodiment> (1-1. Overview of System 1) A system 1 according to an embodiment of the present invention includes a terminal device 10 (an example of an "information processing device").

[0011] The terminal device 10 is realized by, for example, a mobile terminal such as a smartphone owned by an orderer, and is configured to be able to communicate with other devices such as servers via a communication line such as the Internet. A user, as an operator, can operate the terminal device 10 to perform various tasks related to, for example, commercial transactions of goods.

[0012] In the example shown in Fig. 1A, an operator operates the terminal device 10 to take an image of an item P that the operator wishes to put up for auction. After taking the image, the terminal device 10 presents candidates for functions of each service that the operator can use. By selecting a presented function, the operator can perform subsequent work required for that function based on the captured data.

[0013] In this way, by using the system 1 of the present disclosure, it becomes possible to easily carry out the tasks required for commercial transactions involving goods. Each component will be described in detail below.

[0014] (1-2. Hardware configuration of System 1) Fig. 2 is a block diagram showing an example of a hardware configuration of the system 1. The system 1 shown in Fig. 1 includes a terminal device 10, a management server 20, and an external server 30. The terminal device 10, the management server 20, and the external server 30 are communicatively connected via, for example, a network 40. The network 40 is realized by, for example, the Internet and / or a communication network provided by a communication carrier.

[0015] FIG. 2 shows an example in which the system 1 includes only one terminal device 10 for the sake of visibility, but the system 1 may include a plurality of terminal devices 10.

[0016] In this embodiment, a group of multiple devices may be treated as one server. As an example, the method of allocating multiple functions required to realize the management server 20 according to this embodiment may be appropriately determined using one or multiple pieces of hardware in consideration of the processing capacity of the hardware and / or the specifications required for the management server 20.

[0017] The operator can operate the terminal device 10 and use various services provided by the external server 30, for example. The various services include, but are not limited to, transactions for buying and selling goods, arrangements for delivery of goods, requests for repairs of goods, e-mail, SNS (Social Networking Service), management of text data, distribution of videos, and other cloud services. The terminal device 10 is realized by a mobile terminal such as a smartphone, a tablet, VR (Virtual Reality) goggles, and a wearable terminal including a so-called smart watch. The terminal device 10 may also be realized by a stationary PC (Personal Computer) or a laptop PC. In this embodiment, the term "operator" refers to a person who operates the terminal device 10. The term "user" refers to a person who receives the provision of each service, and "user information" refers to information about a user registered in each service. For example, there may be a case where one or more operators operate the terminal device 10 to use the system 1 and receive the provision of each service as one or more users.

[0018] The terminal device 10 includes a communication IF (Interface) 12 , an input device 13 , an output device 14 , a memory 15 , a storage 16 , and a processor 17 .

[0019] The communication IF 12 is an interface for inputting and outputting signals so that the terminal device 10 can communicate with an external device.

[0020] The input device 13 is a device for receiving input operations from an operator (for example, a touch panel, a touch pad, a pointing device such as a mouse, a keyboard, etc.).

[0021] The output device 14 is a device (such as a display and a speaker) for presenting information to an operator.

[0022] The memory 15 is for temporarily storing programs and data to be processed by the programs, and is a volatile memory such as a DRAM (Dynamic Random Access Memory).

[0023] The storage 16 is for storing data, and is, for example, a flash memory or a hard disk drive (HDD).

[0024] The processor 17 is hardware for executing an instruction set written in a program, and is composed of an arithmetic unit, a register, peripheral circuits, and the like.

[0025] The management server 20 is, for example, a device that executes a function of linking the terminal device 10 and the external server 30. The management server 20 manages, for example, data on services provided by the external server 30 and information on users who use the terminal device 10.

[0026] The management server 20 includes a communication IF 22 , an input / output IF 23 , a memory 25 , a storage 26 , and a processor 27 .

[0027] The communication IF 22 is an interface for inputting and outputting signals so that the management server 20 can communicate with external devices.

[0028] The input / output IF 23 functions as an interface with an input device for receiving input operations from an operator and an output device for presenting information to the operator.

[0029] The memory 25 is for temporarily storing programs and data to be processed by the programs, and includes, for example, a volatile memory such as a DRAM.

[0030] The storage 26 is for storing data, and is, for example, a flash memory or a HDD.

[0031] The processor 27 is hardware for executing an instruction set written in a program, and is composed of an arithmetic unit, a register, peripheral circuits, and the like.

[0032] The external server 30 is a device that provides various services. The external server 30 manages, for example, data related to the services provided, and information on users who use the services provided. The hardware configuration of the external server 30 can be configured in the same way as the management server 20.

[0033] (1-3. Functional configuration of terminal device 10) 3 is a diagram showing the configuration of the terminal device 10 constituting the system 1 of the first embodiment. The terminal device 10 is, for example, a mobile terminal compatible with a mobile communication system, such as a smartphone or a tablet. In addition, the terminal device 10 may be, for example, a game console, a wearable device, a PC (Personal Computer), or the like.

[0034] The terminal device 10 is communicatively connected to the management server 20 and the external server 30 via a network such as the Internet. The terminal device 10 is connected to the network by communicating with a communication device such as a wireless base station compatible with communication standards such as 5G and LTE (Long Term Evolution), or a wireless LAN router compatible with wireless LAN (Local Area Network) standards such as IEEE (Institute of Electrical and Electronics Engineers) 802.11.

[0035] As shown in FIG. 2, the terminal device 10 includes an antenna 111, a wireless communication unit 121 corresponding to the antenna 111, an operation reception unit 130 (including a touch-sensitive device 131 and a display 132), an audio processing unit 140, a microphone 141, a speaker 142, a camera 150, a sensor 160, a memory unit 170, and a control unit 180.

[0036] The terminal device 10 also has functions and configurations (such as a battery for storing power, a power supply circuit for controlling the supply of power from the battery to each circuit, etc.) that are not specifically shown in Fig. 3. Each configuration shown by a block in Fig. 3 is electrically connected by a bus or the like.

[0037] The antenna 111 emits a signal generated by the terminal device 10 as a radio wave. The antenna 111 also receives the radio wave from space and provides the received signal to the wireless communication unit 121.

[0038] The wireless communication unit 121 performs modulation / demodulation processing and the like for transmitting and receiving signals via the antenna 111 so that the terminal device 10 can communicate with other wireless devices. The wireless communication unit 121 may be a communication module including a tuner, a received signal strength indicator (RSSI) calculation circuit, a cyclic redundancy check (CRC) calculation circuit, a high-frequency circuit, and the like. The wireless communication unit 121 performs modulation / demodulation and frequency conversion of wireless signals transmitted and received by the terminal device 10, and provides the received signals to the control unit 180.

[0039] The operation reception unit 130 has a mechanism for receiving an input operation from an operator. Specifically, the operation reception unit 130 is configured as a touch screen, and can include a touch-sensitive device 131 and a display 132.

[0040] Touch-sensitive device 131 accepts an input operation by an operator of terminal device 10. Touch-sensitive device 131 detects the touch position of the operator on the touch panel by using, for example, a capacitive touch panel. Touch-sensitive device 131 outputs a signal indicating the touch position of the operator detected by the touch panel to control unit 180 as an input operation.

[0041] Display 132 displays data such as images, videos, and text in response to control by control unit 180. Display 132 can be realized by, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display.

[0042] The audio processing unit 140 modulates and demodulates an audio signal, modulates a signal provided from the microphone 141, and provides the modulated signal to the control unit 180. The audio processing unit 140 also provides the audio signal to the speaker 142. The audio processing unit 140 is realized by, for example, a processor for audio processing. The microphone 141 accepts audio input and provides an audio signal corresponding to the audio input to the audio processing unit 140. The speaker 142 converts the audio signal provided from the audio processing unit 140 into audio and outputs the audio to the outside of the terminal device 10.

[0043] Camera 150 is a device for receiving light with a light receiving element and outputting it as a captured image. Camera 150 may be, for example, a depth camera capable of detecting the distance from camera 150 to a subject to be captured. Camera 150 may also have a function of capturing optically readable information (including one-dimensional codes such as barcodes and two-dimensional codes such as QR codes (registered trademark)) and reading information indicated in the QR codes (registered trademark) based on the captured image.

[0044] The sensor 160 includes various sensors that detect stimuli from the outside world to the terminal device 10, and may include, for example, a voice sensor, a motion sensor, an optical sensor, a biosensor, a tilt sensor, an acceleration sensor, an odor sensor, a taste sensor, and the like. These sensors may be realized by other configurations included in the terminal device 10. For example, the camera 150 may capture moving images and function as a motion sensor that detects a predetermined movement pattern by the operator.

[0045] The storage unit 170 is configured with, for example, a flash memory and a hard disk drive (HDD), and stores data and programs used by the terminal device 10. The storage unit 170 also includes a volatile memory such as a dynamic random access memory (DRAM) for temporarily storing programs and data to be processed by the programs, etc. An example of a database that can be stored in the storage unit 170 will be described later.

[0046] The control unit 180 reads the program stored in the storage unit 170 and executes the instructions included in the program by a processor (hardware for executing an instruction set described in a program, and composed of an arithmetic unit, a register, peripheral circuits, etc.) performing processing according to the program, thereby controlling the operation of the terminal device 10. The control unit 180 may be realized, for example, as an application installed in the terminal device 10 in advance. The control unit 180 operates according to the program to fulfill the functions of an input operation reception unit 1801, a transmission / reception unit 1802, a data extraction unit 1803, an output control unit 1804, and a data management unit 1805.

[0047] The input operation reception unit 1801 performs processing for receiving an input operation of an operator to an input device such as the touch-sensitive device 131. The input operation reception unit 1801 determines the type of touch operation, such as whether the operator's operation is a flick operation, a tap operation, or a drag (swipe) operation, based on information on the coordinates where the operator touches the touch-sensitive device 131 with a finger or the like.

[0048] The transmitting / receiving unit 1802 performs processing for the terminal device 10 to transmit and receive data to and from an external device such as the management server 20 in accordance with a communication protocol.

[0049] The data extraction unit 1803 extracts resource data by generating (or selecting) one or more pieces of resource data to be used for a function provided by an external server, based on an operator's operation accepted by the terminal device 10. Note that the resource data is data necessary for executing the function, and specifically includes image data (including still images and moving images), audio data, text data, and the like.

[0050] The output control unit 1804 performs processing to present information to the operator. The output control unit 1804 performs processing to cause the display 132 to display a display image, processing to cause the speaker 142 to output audio, processing to cause the camera 150 to emit light such as a flash, and the like.

[0051] The data management unit 1805 performs a predetermined process on the data stored in the storage unit 170, based on an operator's operation of the terminal device 10. The process performed by the data management unit 1805 will be described in detail later.

[0052] To use the functions (hereinafter also referred to as service management functions) provided by the system 1, the operator may install a specific application on the terminal device 10. The terminal device 10 may then accept registration of credit card information, information about a payment service (carrier payment) provided by a mobile payment service provider, or information about a payment service (mobile payment) for paying usage fees using a smartphone or the like.

[0053] (1-4. Functional configuration of management server 20) Fig. 4 is a diagram showing the configuration of the management server 20. As shown in Fig. 4, the management server 20 includes a communication unit 201, a storage unit 202, an input / output unit 203, and a control unit 204.

[0054] The communication unit 201 performs processing for the management server 20 to communicate with external devices.

[0055] The storage unit 202 stores data and programs used by the management server 20, and is composed of, for example, a flash memory, a hard disk drive (HDD), etc. The storage unit 202 also includes a volatile memory such as a dynamic random access memory (DRAM) to temporarily store programs and data processed by the programs, etc. An example of a database that can be stored in the storage unit 202 will be described later.

[0056] The input / output unit 203 functions as an interface with an input device for receiving input operations from an operator and an output device for presenting information to the operator.

[0057] The control unit 204 exerts functions shown as various modules by the processor of the management server 20 performing processes according to the programs.

[0058] The reception control module 2041 controls the process in which the management server 20 receives a signal from an external device in accordance with a communication protocol.

[0059] The transmission control module 2042 controls the process in which the management server 20 transmits a signal to an external device in accordance with a communication protocol.

[0060] The data management module 2043 performs a predetermined process on the data stored in the storage unit 202, based on an operator's operation of the terminal device 10. The process performed by the data management module 2043 will be described in detail later.

[0061] (1-5. Functional configuration of external server 30) 5 is a diagram showing the configuration of the external server 30. As shown in FIG. 5, the external server 30 includes a communication unit 301, a storage unit 302, an input / output unit 303, and a control unit 304.

[0062] The communication unit 301 performs processing for the external server 30 to communicate with external devices.

[0063] The storage unit 302 stores data and programs used by the external server 30, and is composed of, for example, a flash memory, a HDD (Hard Disc Drive), etc. The storage unit 302 also includes a volatile memory such as a DRAM (Dynamic Random Access Memory) to temporarily store programs and data processed by the programs, etc.

[0064] The input / output unit 303 functions as an interface with an input device for receiving input operations from an operator and an output device for presenting information to the operator.

[0065] The control unit 304 exerts functions shown as various modules by the processor of the external server 30 performing processes according to the programs.

[0066] The reception control module 3041 controls the process in which the external server 30 receives a signal from an external device in accordance with a communication protocol.

[0067] The transmission control module 3042 controls the process in which the external server 30 transmits a signal to an external device in accordance with a communication protocol.

[0068] The function execution module 3043 performs a predetermined process for executing a function of a service provided by the external server 30, based on an operator's operation of the terminal device 10. The process performed by the function execution module 3043 will be described in detail later.

[0069] (1-6. Data Structure) 6 is a diagram showing an example of the data structures of the service database 2021, the service function database 2022, the service action database 2023, the user database 2024, the provided function database 2025, and the action database 2026 stored in the storage unit 202 of the management server 20. Note that the data structures shown below are merely examples, and the application of the technical scope of the present disclosure is not limited to these data structures.

[0070] The service database 2021 is a database that stores information about each service linked to the system 1. Each record in the service database 2021 includes an item "service ID", an item "service name", an item "content", and the like.

[0071] The item "service ID" is information for identifying each service with which the system 1 is affiliated.

[0072] The item "service name" is information on the name of each service.

[0073] The item "content" is information regarding the content of each service.

[0074] The data management module 2043 of the management server 20 adds a record to the service database 2021 when the system 1 cooperates with each service.

[0075] The service function database 2022 is a database that stores information about functions provided by each service. Each record in the service function database 2022 includes an item "service function ID", an item "service ID", an item "function content", and the like.

[0076] The item "service function ID" is information for identifying a function provided in each service.

[0077] The item "service ID" is information for identifying each service that provides the function, and the item "service ID" of the service database 2021 is held as an external key.

[0078] The item "function content" is information about the content of the function.

[0079] The data management module 2043 adds a record to the service function database 2022 when the system 1 cooperates with each service.

[0080] The service action database 2023 is a database that stores information regarding which action (the first action by the operator) is used as a trigger to present a candidate for a function provided by each service used by the user, and which action (the second action by the operator) is used to execute or transmit the function. An action is a combination of an operation performed by the operator on the terminal device 10 and processing of data. Specifically, the action includes, but is not limited to, taking an image (including a still image and a video) by a display operation such as tapping, generating voice data, generating text, and selecting a file (including an image file, a voice file, a character string file, etc.). In addition, an image may be generated or a file may be selected by uttering a pattern voice corresponding to a voice sensor, such as breathing, or by performing an action corresponding to a motion sensor, such as closing the eyes. Each record in the service action database 2023 includes an item "service function ID", an item "trigger AID", an item "execution AID", etc.

[0081] The item "service function ID" holds the item "service function ID" of the service function database 2022 as a foreign key.

[0082] The item "trigger AID" is information for identifying the first action performed by the operator to extract resource data, and the item "AID" of the action database 2026 is held as an external key.

[0083] The item "execution AID" is information for identifying a second action performed by an operator user to execute transmission of resource data, and the item "AID" of the action database 2026 is held as an external key.

[0084] The data management module 2043 adds a record to the service action database 2023 when the system 1 cooperates with each service.

[0085] The user database 2024 is a database that stores information about users.

[0086] Each record of the user database 2024 includes an item "user ID", an item "service ID", an item "email address", an item "password", and the like.

[0087] The item "user ID" is information for identifying a user who uses each service.

[0088] The item "service ID" is information for identifying each service used by the user, and the item "service ID" of the service database 2021 is held as an external key.

[0089] The item "email address" is information about the user's email address, which can be used for authentication purposes (hereinafter, also referred to as "login") when using each service. However, information other than an email address may be used when a user logs in.

[0090] The item "password" is information about the password used by the user when logging in to each service. However, information other than a password may be used when logging in by the user.

[0091] The data management module 2043 adds a record to the service action database 2023 when the user registers user information for each service in the system 1 .

[0092] The provided function database 2025 is a database that stores information regarding the correspondence between user information of each user and available service functions. Each record of the provided function database 2025 includes an item "user ID" and an item "service function ID", etc.

[0093] The item "user ID" holds the item "user ID" of the user database 2024 as a foreign key.

[0094] The item "service function ID" holds the item "service function ID" of the service function database 2022 as a foreign key.

[0095] The data management module 2043 adds a record to the provided function database 2025 when the user registers user information for each service in the system 1 .

[0096] The action database 2026 is a database that stores information about actions taken by an operator by operating the terminal device 10. Each record in the action database 2026 includes an item "AID", an item "data processing", an item "operation content", and the like.

[0097] The item "AID" is information for identifying an action to be performed by an operator.

[0098] The item "data processing" is information about the content of data processing executed by the action.

[0099] The item "operation content" is information about the operation content performed by the operator on the terminal device 10 in the action.

[0100] The contents of the action database 2026 are preset. However, if the specifications allow the operator to register additional actions, the data management module 2043 adds a record to the action database 2026 when the operator registers the action.

[0101] FIG. 7 is a diagram showing an example of the data structures of the first action database 1701, the second action database 1702, and the display action database 1703 stored in the storage unit 170 of the terminal device 10. As shown in FIG.

[0102] The first action database 1701 is a database that stores information about a first action performed by an operator to extract resource data. The first action is an action performed by an operator by operating the terminal device 10 to extract resource data. Each record in the first action database 1701 includes an item "first AID" and an item "AID", etc.

[0103] The item "first AID" is information for identifying the first action taken by the operator to extract resource data.

[0104] The item "AID" is information for identifying an action corresponding to the first action, and the item "AID" of the action database 2026 is held as an external key.

[0105] The contents of the first action database 1701 are set according to the terminal device 10. However, if the specifications are such that an additional first action can be registered by the operator, the data management unit 1805 of the terminal device 10 adds a record to the first action database 1701 at the timing when the operator registers it.

[0106] The second action database 1702 is a database that stores information about second actions for executing the functions of each service. The second action is an action by an operator for executing the functions of each service using resource data. Specifically, functions executed by the second action include, but are not limited to, sending resource data, synthesizing a new file (e.g., a CG (Computer Graphics) file) using resource data, detecting and / or registering a fingerprint of something, receiving an item, and making a payment. Each record in the second action database 1702 includes an item "second AID", an item "AID", an item "user ID", an item "service function ID", and the like.

[0107] The item "second AID" is information for identifying a second action for executing the function of each service.

[0108] The item "AID" is information for identifying an action corresponding to the first action, and the item "AID" of the action database 2026 is held as an external key.

[0109] The item "user ID" holds the item "user ID" of the user database 2024 as a foreign key.

[0110] The item "service function ID" is information about the function executed by the second action, and the item "service function ID" of the service function database 2022 is held as a foreign key.

[0111] When an operator registers information about a user of each service in the system 1, the data management unit 1805 of the terminal device 10 retrieves a record of the service function corresponding to the user from the provided function database 2025 of the management server 20, and adds the record to the second action database 1702.

[0112] The display action database 1703 is a database that stores information regarding the priority order when the first action is displayed on the display 132. Each record of the display action database 1703 includes an item "display No.", an item "first AID," and the like.

[0113] The item "Display No." is information regarding the priority order when the first action is displayed.

[0114] The item "first AID" holds the item "first AID" of the first action database 1701 as a foreign key.

[0115] The data management unit 1805 of the terminal device 10 adds a record to the display action database 1703 at the timing when the operator operates the terminal device 10 to set the display order of the first action.

[0116] (1-7. Screen example) Examples of screens on the terminal device 10 will be described with reference to FIGS.

[0117] FIG. 8 is a diagram illustrating an example of a screen displayed on the terminal device 10 when the first and second actions are accepted.

[0118] In the screen example (A), an image of an item (specifically, a desk) photographed by the operator by activating the camera 150 is displayed on the terminal device 10. In this way, the operator photographs an image of the item as the first action, and the terminal device 10 accepts resource data (image data in this example). Alternatively, the operator may hold the camera 150 over the item as the first action, and the terminal device 10 may accept the resource data. In this way, the resource data may be an image of the item (including at least a part of it), or may be identification information stored in an image, a two-dimensional code, a three-dimensional code, a fingerprint of an object, or other information. In addition, the resource data may be generated by the first action, or may be selected from data stored in the storage unit 170 or the like by the first action.

[0119] As an example, the selection dial 60 has a central portion 60a and an annular portion 60b, and is configured so that the operator can rotate the annular portion 60b by tracing it with a finger, etc. This rotation operation allows switching between a plurality of first actions, and the selected first action can be executed by pressing the central portion 60a.

[0120] In the screen example (B), the terminal device 10 displays function candidates 61 to be executed by the second action. The operator can select which user will use the function as the second action candidate 61 based on the user icon 61a. The second action candidates 61 may be displayed in multiple numbers, or only one may be displayed. In addition, multiple different function candidates may be displayed from one resource data. Specifically, one or multiple functions may be presented as candidates to be executed by one user, and one or multiple identical or different functions may be displayed as candidates to be executed by other users. With such specifications, for example, a sender of an item and a recipient of the item can each execute different functions (for example, a function to arrange delivery of the item and a function to receive the item) for the same resource data. The operator can select one or multiple of the second action candidates 61 by an operation such as pressing a radio button 61b. In addition, when multiple function candidates are identified, the second action candidate may be limited to a "selection operation" action. Furthermore, when one function candidate is identified, the second action for executing the function may be not only a selection operation but also an image capture operation, a text input operation, or the like.

[0121] In the screen example (C), completion of the procedure as the second action is displayed on the terminal device 10. In this manner, the operator can perform a first action for accepting resource data used for a function provided in each service and a second action for executing the function by performing a predetermined operation on the terminal device 10.

[0122] 9 is a diagram for explaining an example of use of the selection dial 60 displayed on the terminal device 10. The operator can rotate the selection dial 60 to switch between the screen registered as the first action and / or other screens. Examples of screens (A) to (D) are shown below.

[0123] The screen example (A) is a screen on which the operator captures an image of an item or the like using the camera 150 of the terminal device 10 as a first action. The screen example (B) is a screen on which the operator inputs text and generates a memo as a first action.

[0124] The screen example (C) is a screen on which a list of received product information is displayed on the terminal device 10. The screen example (D) is a screen on which a list of various received notifications is displayed.

[0125] Also, for example, information such as a membership card uniquely generated by the terminal may be displayed. Each of these screens may be displayed in the order of priority registered by the operator, or the learning model may weight the screens based on the operator's usage history and user information registered in each service (age, sex, address, occupation, preferences, etc.), and the screens may be displayed in the order based on the weighting.

[0126] In this way, a plurality of screens can be switched and displayed according to the rotational position of the annular portion 60b of the selection dial 60. The operator can select a preregistered function by rotating the annular portion 60b of the selection dial 60, and execute the function on the selected screen by pressing the center portion 60a.

[0127] 10 is a diagram illustrating an example of a management screen displayed on the terminal device 10. In screen example (A), a linked service management selection icon 63 and a linked function management selection icon 64 are displayed as a setting screen. When the operator selects the linked service management selection icon 63, screen example (B) is displayed. When the operator selects the linked function management selection icon 64, screen example (C) is displayed.

[0128] The linked service management screen of screen example (B) allows the operator to register each service available for use. Specifically, by pressing an add button 65, the operator can select a service that he or she wishes to use from among the available services registered in the service database 2021 of the management server 20. The service selected by the operator is displayed in a linked service icon 66.

[0129] In the display function management screen of the screen example (C), the operator can register the display order of the first action and the second action that he / she wants to display preferentially. When the operator registers the display order of the first action, a record is added to the display action database 1703, and the first action is displayed in the display order. As described above, the display of the candidates of the first action and the second action may be weighted by the learning model based on the data type of the resource data (file format, data type, data capacity, etc.) and the user information registered in each service (age, sex, occupation, frequency of use of the user), etc., so that appropriate candidates are displayed in an appropriate order. In accordance with this, by allowing the operator to appropriately set the order, a stronger weighting may be applied to the weighting set by the learning model, and a display order that matches the operator's preferences may be realized.

[0130] (1-8. Processing flow) The process flow of the system 1 will be described with reference to FIG.

[0131] In step S1110, the control unit 180 of the terminal device 10 performs a first action execution process. The first action execution process will be described in detail later.

[0132] In step S1120, the control unit 180 of the terminal device 10 acquires candidates for the second action. As an example, the control unit 180 may acquire, as candidates, second actions linked to the first action performed by the operator, with reference to the action link database 1704. Furthermore, the control unit 180 may acquire candidates for the second action based on the type of resource data (file format, file size, data type, etc.) and / or user information.

[0133] In step S1130, the control unit 180 of the terminal device 10 displays on the display 132 the candidates for the second action acquired in step S1120.

[0134] In step S1140, the control unit 180 of the terminal device 10 performs a second action execution process. The second action execution process will be described in detail later.

[0135] The flow of the first action execution process (S1110) will be described with reference to FIG.

[0136] In step S1111, the control unit 180 of the terminal device 10 starts up the camera 150 based on an operation by the operator.

[0137] In step S 1112 , the control unit 180 of the terminal device 10 displays the selection dial 60 on the display 132 .

[0138] In step S1113, the control unit 180 of the terminal device 10 can switch the first action by operating the selection dial 60 by the operator.

[0139] In step S1114, the control unit 180 of the terminal device 10 causes the camera 150 to capture an image.

[0140] The flow of the second action execution process (S1140) will be described with reference to FIG.

[0141] In step S1141, the control unit 180 of the terminal device 10 accepts an operation for the second action (for example, a selection operation).

[0142] In step S1142, the control unit 180 of the terminal device 10 transmits an instruction to execute a function associated with the second action selected by the operator to the external server 30. In step S3141, the control unit 304 of the external server 30 receives the instruction to execute the function.

[0143] In step S3142, the control unit 304 of the external server 30 inquires of the management server 20 about the terminal device 10 that transmitted the instruction to execute. In step S2141, the control unit 204 of the management server 20 refers to the user database 2024 and transmits user information registered in each service.

[0144] In step S3143, the control unit 304 of the external server 30 determines whether or not the user has authority to execute the function, based on the user information received from the management server 20. In step S3144, the control unit 304 of the external server 30 executes the function. Note that the authority determination process may be performed by the management server 20.

[0145] In step S3145, the control unit 304 of the external server 30 transmits the processing result of the function in the external server 30. In step S1143, the control unit 180 of the terminal device 10 receives the processing result of the function in the external server 30.

[0146] In step S1144, the control unit 180 of the terminal device 10 displays on the display 132 the processing result of the relevant function in the external server 30.

[0147] In this manner, the system 1 of the present disclosure executes the steps of accepting resource data to be used for a specific function provided in one or more services through a first action of an operator, identifying at least one function associated with a user in the service, and presenting candidates for a second action for executing the identified function to the operator.

[0148] With this configuration, the operator does not need to install the applications provided by each service, and can easily perform the required terminal operations. In addition, since resource data is generated and function candidates for each service are presented by one action, the operator can complete the procedure with fewer operations. Furthermore, the service provider does not need to develop each application individually, and can provide a variety of services by linking with one application installed by the operator (i.e., the application provided by the system 1).

[0149] In this manner, in the system 1 according to the present disclosure, an action for generating or selecting resource data (first action) and an action for executing the resource data (second action) can be performed consecutively.

[0150] Furthermore, functions identified as functions using resource data are basically associated with each user by the service provider, and may not depend on whether the service is already linked or not linked by the operator, and may include functions provided in services already linked with user information and functions provided in services not linked with user information. In this case, the management server 20 may be configured to suggest functions of each service that are not registered by the operator by referring to the service database 2021 and / or the service function database 2022. With such a configuration, the operator can use the functions of the service without performing work for linking by himself.

[0151] In addition, functions provided in a service that is already linked with user information may include functions that are authorized to be executed by the associated user information and functions that are not authorized to be executed by the user information. With this configuration, it is possible to guide the user to a predetermined procedure for granting authority (for example, a registration procedure for a paid member) for functions that are not authorized, eliminating the cumbersome procedures for the operator.

[0152] In addition, the step of judging whether the resource data is of a data type capable of executing a function is further executed. With this configuration, it is possible to perform a series of processes to determine whether the function can be executed.

[0153] Also, based on at least one of user information, data type of resource data, and information obtained from the resource data (such as tag information added by image detection technology), candidates for second actions for executing functions other than the pre-registered functions are presented to the operator. Specifically, candidates for second actions for executing functions that have no record in the second action database 1702 of the terminal device 10 but have a record in the service function database 2022 of the management server 20 are presented. With this configuration, it is possible to reduce the operator's effort in selecting a required second action.

[0154] In addition, the element for selecting the first action is formed in a ring shape and configured to be rotatable. With such a configuration, the operation method becomes easier for the operator to understand and operability is improved.

[0155] Furthermore, an element (e.g., selection dial 60) displayed for executing the first action or the second action is configured to be movable based on an operator's operation (e.g., a touch operation on a touch panel). In this case, the central portion 60a and the annular portion 60b of the selection dial 60 may be configured to move together on the display 132, or the central portion 60a and the annular portion 60b may be configured to move separately. With this configuration, for example, when operating a camera to capture an image as the first action, the operator can freely change the position at which the camera is operated, which enables operation with one hand, for example.

[0156] In addition, the element for selecting the first action is configured to be operable in response to a voice pattern generated by the operator or a motion pattern of an action performed by the operator. With this configuration, the terminal device 10 can be operated even in a situation where, for example, both hands are full.

[0157] The system further includes a function registration step of accepting registration of a function associated with the user information through an operation by the operator. With this configuration, the operator can register and use a desired function.

[0158] In addition, the system accepts the operator's selection of the output second action candidates, and executes the processing of the function associated with the selected second action. With this configuration, the process from the selection of the second action to the execution of the function can be performed in a single flow.

[0159] Also, a new available function may be added in response to an action performed by an operator. For example, a receiving function for receiving an item may be added by performing a purchase transaction as the second action, and a record for that function may be added to the database of the terminal device 10 or the management server 20.

[0160] (1-9. Modifications) Modifications of the above embodiment will be described below. The selection dial 60 may accept voice operations. For example, the annular portion 60b may rotate like a windmill when the operator blows on it, and the displayed first action may be switched. Also, the first action may be executed by the sound generated when the operator blows strongly on the central portion 60a, as if the central portion 60a was pressed.

[0161] In the above embodiment, the resource data for the external server 30 to execute a function is transmitted to the external server 30 by the terminal device 10, but the present invention is not limited to this specification. For example, the management server 20 may be configured to estimate the resource data based on the data received from the terminal device 10. In this case, a machine-learned model may be stored in the storage unit 202 of the management server 20, and the management server 20 may estimate the resource data (e.g., an image of the entire item) from the data received (e.g., an image showing a part of the item) using the machine-learned model.

[0162] In addition, in the above case, the management server 20 may be configured to determine whether the data received from the terminal device 10 contains resource data or whether the resource data can be estimated from the received data, and if it determines that this is not possible, to issue an error notification or the like so as to send further data to the terminal device 10.

[0163] Furthermore, the management server 20 may execute a recommendation process to propose a function (which may be a function in the same service or a function in a different service) different from the second action instructed to be executed by the operator. In this case, the storage unit 202 of the management server 20 may store user actions for service functions. Then, the management server 20 may generate a trained model by machine learning for a combination of the resource data in the history and the specified function, so that the management server 20 proposes an optimal function for the resource data received from the terminal device 10.

[0164] In the above case, the management server 20 may further propose an optimal specification based on information about the operator who operated the terminal device 10. In this case, the management server 20 may propose an optimal function based on information about the operator acquired from the operation history of the terminal device 10.

[0165] In the above embodiment, the first action is switched by rotating the annular portion 60b of the selection dial 60 displayed on the display 132 of the terminal device 10, but the present invention is not limited to this specification. For example, the operator may use a voice pattern or a motion pattern to switch the first action (and the second action) to be selected.

[0166] In the above embodiment, the processing of the system 1 is executed by the terminal device 10, the management server 20, and the external server 30, but the present invention is not limited to this example. For example, the processing of the management server 20 may be executed by the terminal device 10, or the processing of the management server 20 and the external server 30 may be executed by one server. Furthermore, the terminal device 10 and the management server 20 may be connected to each other without going through the network 40.

[0167] The terminal device 10 may be designed to store the resource data generated when the first action is executed in the storage unit 170. Furthermore, the terminal device 10 may be designed to have a transmission reservation function for temporarily storing the generated resource data or the contents of the processing from the selection of the second action to the connection to the external server 30 in the storage unit 170 in an environment where a communication connection to the network 40 is not possible, and transmitting the resource data to the external server 30 (or the management server 20) after a communication connection is established. In this way, the operator can complete the work using the terminal device 10, improving the convenience of the operator.

[0168] In the above embodiment, the management server 20 performs authentication processing using the email address and password stored in the user database 2024, but instead of using a fixed password, for example, the management server 20 may issue a one-time password (a randomly generated password that is valid only for a certain period of time) and perform authentication using the one-time password. Also, the authentication method is not limited to passwords, and authentication may be performed in a manner different from a password, such as unique identification information and biometric authentication.

[0169] Furthermore, when the external server 30 inquires about user information (step S3142), the management server 20 may be configured to determine whether or not a function can be executed based on not only the user information but also the data type of the resource data. In this case, for example, the data type of the resource data that can be accepted for each function of each service may be defined in the service function database 2022 or the like, and when resource data different from the defined data type is accepted from the terminal device 10, it may be determined that the function cannot be executed.

[0170] Also, the processing executed by the first action may be stored, and an instruction to repeat a series of processing of the second action from the acceptance of the first action to the display of the execution result of the function on the terminal device 10 may be received from the operator. For example, as shown in FIG. 14, a slide button 68 for storing a series of processing may be provided on a screen for selecting a second action. By switching ON / OFF of the slide button 68 by the operator, it is possible to store the series of processing from the acceptance of the first action to the display of the execution result of the second action when the slide button is ON. The operator can confirm that his / her operation is being stored by checking the recording icon 67 indicating that storage is in progress. With such a function, for example, when an image capturing operation of a large number of two-dimensional codes for identifying goods is to be performed as the first action, and a similar service function (for example, an inventory function) is to be performed as the second action, the second action can be omitted by storing the consecutive second action, and the desired work can be performed by consecutive operations of the first action. Note that the first action may be omitted and the second action may be consecutively operated, or the processing of the second action may be stored and an instruction to repeatedly execute the first action may be received.

[0171] Also, a function for grouping a plurality of user IDs may be implemented. For example, as shown in FIG. 15, an item "management account" and an item "management password" may be added to the user database 2024. The management account is an account for integrating a plurality of grouped user IDs, and various functions can be executed as a plurality of user IDs by logging in to the system 1 with the management account. The management password is a password for logging in to the system 1 with the management account. Note that the mode of the management password is not limited as long as it can be authenticated at the time of login. By grouping a plurality of user IDs in this way and making them available, it is possible to easily use a wider variety of services. Also, even when a terminal device other than the terminal device 10 is used, it is possible to use the history of services registered up to that point by using the management account.

[0172] As shown in FIG. 16, the terminal device 10 may take various forms. For example, as shown in screen example (A), the terminal device 10 may be a foldable smartphone. In this case, the selection dial 60 is disposed on the folded hand side, thereby ensuring operability and visibility. As shown in screen example (B), the terminal device 10 may be realized as a watch-type wearable terminal. In this case, the selection dial 60 is formed in a semi-arc shape, thereby ensuring operability and visibility. As shown in screen example (C), the terminal device 10 may be realized as VR goggles (or AR goggles). In this case, the selection dial 60 may be designed to receive rotation operations in three-dimensional directions, thereby improving operability.

[0173] <2. Second embodiment> Hereinafter, the system 1 according to the second embodiment of the present disclosure will be described with reference to FIGS.

[0174] (2-1. Overview) A system 1 according to an embodiment of the present invention includes a plurality of terminal devices 10 (an example of an "information processing device") and a management server 20 (an example of an "information processing device").

[0175] The terminal device 10 is realized by a mobile terminal such as a smartphone owned by an orderer, and is configured to be able to communicate with the management server 20 via a communication line such as the Internet. The management server 20 manages transactions related to the purchase of goods. Examples of goods include, but are not limited to, furniture, home appliances, electronic devices, daily necessities, clothing, vehicles, etc. A user can perform various tasks related to commercial transactions of goods by operating the terminal device 10.

[0176] In the example shown in Fig. 17A, seller A operates the terminal device 10 to take an image of an item P that he or she wishes to put up for sale, and transmits the image data and the details of the necessary work to the management server 20. The management server 20 performs image processing on the received image data and registers the product information.

[0177] In the example shown in FIG. 17B, purchaser B purchases item P by operating terminal device 10. Management server 20 notifies seller A that item P has been purchased. Seller A photographs delivery item Q in which item P is packed, and transmits the image data and the details of the necessary work to management server 20. Management server 20 processes the received image data and registers information about delivery item Q. Delivery item Q is delivered to purchaser B while being managed by system 1.

[0178] In this way, by using the system 1 of the present disclosure, it becomes possible to easily perform the tasks required for commercial transactions involving goods based on the image data of the goods, while managing the status of the transaction. Each component will be described in detail below.

[0179] (2-1. Configuration of Management Server 20) 18 is a diagram showing the configuration of the management server 20. As shown in FIG. 18, the management server 20 includes a communication unit 201, a storage unit 202, an input / output unit 203, and a control unit 204.

[0180] The communication unit 201 performs processing for the management server 20 to communicate with external devices.

[0181] The storage unit 202 stores data and programs used by the management server 20, and is composed of, for example, a flash memory, a hard disk drive (HDD), etc. The storage unit 202 also includes a volatile memory such as a dynamic random access memory (DRAM) to temporarily store programs and data processed by the programs, etc. As an example, the storage unit 202 stores a user information database 281, a product information database 282, an order information database 283, etc.

[0182] The user information database 281 is a database for holding information on each user in the system 1. Details will be described later.

[0183] The product information database 282 is a database for holding information on items (hereinafter also referred to as products) offered for sale on the system 1. Details will be described later.

[0184] The order information database 283 is a database for storing information on orders for products from users, as will be described in detail later.

[0185] The input / output unit 203 functions as an interface with an input device for receiving input operations from a user and an output device for presenting information to the user.

[0186] The control unit 204 exerts functions shown as various modules by the processor of the management server 20 performing processes according to the programs.

[0187] The reception control module 2031 controls the process in which the management server 20 receives a signal from an external device in accordance with a communication protocol.

[0188] The transmission control module 2032 controls the process in which the management server 20 transmits a signal to an external device in accordance with a communication protocol.

[0189] The image processing module 2033 performs predetermined image processing on the image data transmitted by the user from the terminal device 10. The image processing performed by the image processing module 2033 will be described in detail later.

[0190] The procedure execution module 2034 uses the image that has been subjected to image processing to execute a procedure necessary for the work that the user has input to the terminal device 10. The procedure executed by the procedure execution module 2034 will be described in detail later.

[0191] (2-2. Data Structure) FIG. 19 is a diagram showing an example of the data structure of the user information database 281, the product information database 282, and the order information database 283 stored in the storage unit 202 of the management server 20. As shown in FIG.

[0192] As shown in FIG. 19, each record of the user information database 281 includes an item "user ID", an item "user name", an item "attribute information", an item "registration date", and the like.

[0193] The item "user ID" is information for identifying each user.

[0194] The item "user name" is the name set by the user.

[0195] The item "attribute information" is information indicating basic information of a user, and specifically includes items "age," "gender," "address," etc. For example, when a user registers as a member of a commercial transaction service provided by the system 1, input of such user information may be accepted from the user at the user's discretion. In addition, the management server 20 may acquire the user's basic information from an SNS by linking with the user's SNS account.

[0196] The item “registration date” is information on the date on which each user registered for the commercial transaction service provided by the system 1.

[0197] The management server 20 adds a record to the user information database 281 as each user registers with the commercial transaction service.

[0198] As shown in FIG. 19, each record in the product information database 282 includes an item "product ID", an item "product name", an item "seller", an item "price", an item "image", an item "status", and the like.

[0199] The item "product ID" is information that identifies a product offered for sale in the commercial transaction service.

[0200] The item "product name" is the name of the product.

[0201] The item "sell user" is information for identifying the user who put up the item for sale, and the item "user ID" of the user information database 281 is held as an external key.

[0202] The item "price" is information about the price of the product.

[0203] The item "image" holds image data of the product and / or data obtained by performing image processing on the image data.

[0204] The item "status" is information that indicates the state of the product in a transaction related to the purchase of the product. Specifically, the item "status" can take values ​​such as "on sale", "in delivery", and "delivered".

[0205] When each user puts up a product for sale on the commercial transaction service, the management server 20 adds a record to the product information database 282. Also, when each user performs a predetermined operation on the product, the management server 20 updates the product information database 282.

[0206] As shown in FIG. 19, each record in the order information database 283 includes an item "order ID", an item "product ID", an item "ordering user", an item "order date", and an item "payment method".

[0207] The item "order ID" is information for identifying an order from a user in a commercial transaction service.

[0208] The item "product ID" is information for identifying the product to be ordered, and the item "product ID" of the product information database 282 is held as an external key.

[0209] The item "ordering user" is information for identifying the user who placed the order, and the item "user ID" of the user information database 281 is held as an external key.

[0210] The item "order date" is information on the date of the order.

[0211] The item "payment method" is information about the payment method for the order.

[0212] The management server 20 adds a record to the order information database 283 as each user places an order with the commercial transaction service.

[0213] (2-3. Screen example) FIG. 20 is a diagram showing an example of a screen of the terminal device 10 of the present disclosure.

[0214] The screen example (A) of FIG. 20 is a diagram showing a state in which a screen for accepting operations to be performed on a transaction from the user is displayed on the terminal device 10.

[0215] As shown in the example screen (A), the user selects a task selection icon 51 to input a task to be performed regarding the transaction from among a plurality of tasks displayed as candidates on the terminal device 10. The details of the plurality of tasks will be described later.

[0216] The screen example (B) of FIG. 20 is a diagram showing a state in which a screen for accepting image data of an article from a user is displayed on the terminal device 10.

[0217] As shown in the example screen (B), the user can take an image of an item using the camera 150 of the terminal device 10 by selecting the image selection icon 52. Alternatively, the user can select image data that is stored in advance in the library of the storage unit 170 of the terminal device 10 by selecting the image selection icon 52.

[0218] The screen example (C) of FIG. 20 is a diagram showing a situation in which a confirmation screen 53 for confirming whether or not the user wants to carry out the subsequent procedure using the image data selected by the user is displayed on the terminal device 10.

[0219] As shown in example screen (C), by pressing “Yes” or “No” on confirmation screen 53, the user can perform image processing on the image data selected by the user and determine whether or not to proceed with the subsequent procedure.

[0220] The screen example (D) of FIG. 20 is a diagram showing a situation in which a notification screen 54 is displayed on the terminal device 10 to notify that the procedures required for the work to be performed regarding the transaction have been completed.

[0221] As shown in the example screen (D), by checking the notification screen 54, the user can understand that the procedures necessary for the work have been completed.

[0222] (2-4. Contents of work and procedures) With reference to Fig. 21, the tasks performed in relation to the transaction of goods and the procedures required for the tasks will be described. Tables (A) to (C) in Fig. 21 list the tasks performed in relation to the transaction of goods, image data used in the tasks, image processing (selectively) performed on the image data, information (concrete information and abstract information) acquired from the image data, and procedures executed in the tasks. Each task will be described below.

[0223] (2-4-1. Product registration) In the process of registering a product, as shown in Table (A), the user (seller) inputs image data of the product's exterior into the terminal device 10. The user also inputs product registration as a transaction-related task. The terminal device 10 transmits the image data entered by the user and the details of the task to the management server 20.

[0224] The image processing module 2033 of the control unit 204 of the management server 20 executes, for example, photogrammetry processing and / or AI (Artificial Intelligence) / ML (Machine Learning) search processing as image processing on the image data.

[0225] For example, when photogrammetry processing is performed, a 3D model is generated based on the image data of the item. The procedure execution module 2034 of the management server 20 uses the image data that has been subjected to image processing to register the item in the product information database 282 and create a display page for the item.

[0226] (2-4-2. Registration of unique information) In the unique information registration process, as shown in Table (A), the user (seller) inputs image data of characteristic parts of the product into the terminal device 10. The user also inputs the registration of unique information as a transaction-related task. The terminal device 10 transmits the image data input by the user and the details of the task to the management server 20.

[0227] The image processing module 2033 of the control unit 204 of the management server 20 executes, for example, Fingerprint of Things processing, and / or AI / ML search processing, and / or AR (Augmented Reality) processing as image processing on the image data.

[0228] As an example, when the Fingerprint of the Object process is executed, a pattern or the like on the surface of the object is acquired as unique identification information based on image data of characteristic parts of the object. The procedure execution module 2034 of the management server 20 uses the image data that has been subjected to the image processing to register the image data in the product information database 282.

[0229] (2-4-3. Shipping) In the task of shipping a product, as shown in Table (B), the user (sender) inputs image data of characteristic parts of the product into the terminal device 10. The user also inputs the shipping of the product as a task to be performed in relation to the transaction. The terminal device 10 transmits the image data input by the user and the details of the task to the management server 20.

[0230] The image processing module 2033 of the control unit 204 of the management server 20 executes Fingerprint of Object processing as image processing for the image data.

[0231] As an example, when the Fingerprint of the Object process is executed, a pattern or the like on the surface of the object is acquired as unique identification information based on image data of a characteristic part of the object. The procedure execution module 2034 of the management server 20 updates the product information database 282 using the image data that has been subjected to the image processing.

[0232] (2-4-4. Packaging registration) In the registration work of the package, as shown in Table (B), the user (packer) inputs image data of the exterior of the package into the terminal device 10. The user also inputs the registration of the package as a work to be performed regarding the transaction. The terminal device 10 transmits the image data input by the user and the contents of the work to the management server 20.

[0233] The image processing module 2033 of the control unit 204 of the management server 20 executes, for example, Fingerprint of Things processing and / or photogrammetry processing as image processing on the image data.

[0234] As an example, when the Fingerprint of the Object process is executed, the surface pattern of the package is acquired as unique identification information based on the image data of the characteristic part of the package. The procedure execution module 2034 of the management server 20 updates the product information database 282 using the image data that has been subjected to the image processing.

[0235] (2-4-5. Receiving) In the shipping task, as shown in Table (C), the user (receiver) inputs image data of the characteristic parts of the product into the terminal device 10. The user also inputs receipt of the product as a task to be performed in relation to the transaction. The terminal device 10 transmits the image data input by the user and the details of the task to the management server 20.

[0236] The image processing module 2033 of the control unit 204 of the management server 20 executes, for example, Fingerprint of Object processing as image processing on the image data.

[0237] As an example, when the Fingerprint of an Object process is executed, the surface pattern of the object is acquired as unique identification information based on the image data of the characteristic part of the object. The procedure execution module 2034 of the management server 20 uses the image data that has been subjected to the image processing to determine whether the image data matches the information registered in the product information database 282.

[0238] (2-4-6. Inspection) In the inspection of goods, as shown in Table (C), the user (inspector) inputs image data of the exterior of the goods into the terminal device 10. The user also inputs the inspection of the goods as a task to be performed in relation to the transaction. The terminal device 10 transmits the image data input by the user and the details of the task to the management server 20.

[0239] The image processing module 2033 of the control unit 204 of the management server 20 executes, for example, AI / ML search processing as image processing on the image data.

[0240] As an example, when an AI / ML search process is executed, the image processing module 2033 searches for an image similar to the image data of the item. The procedure execution module 2034 of the management server 20 uses the searched image as training data to inspect the presence or absence of defects in the item to be inspected. The procedure execution module 2034 updates the product information database 282.

[0241] (2-5. Processing flow) Hereinafter, the flow of processing by the terminal device 10 and the management server 20 will be described with reference to FIG.

[0242] In step S611, the terminal device 10 accepts an operation to input work content from the user. Specifically, the user inputs the work content by selecting a corresponding work from among a plurality of work selection icons 51 presented on the terminal device 10.

[0243] In step S612, the terminal device 10 accepts an operation to input image data of an article from the user. Specifically, the user selects the image selection icon 52 displayed on the terminal device 10 to input image data of the article.

[0244] In step S621, the management server 20 receives the work content and image data of the item input by the user.

[0245] In step S622, the image processing module 2033 of the management server 20 performs image processing on the image data of the item in accordance with the input work content.

[0246] In step S623, the procedure execution module 2034 of the management server 20 executes the procedure required for the work using the image data that has been subjected to image processing.

[0247] In step S613, the terminal device 10 notifies the user that the procedure required for the work has been performed by the management server 20. Specifically, the terminal device 10 notifies the user that the procedure required for the work has been performed by displaying the notification screen 54.

[0248] As described above, in this embodiment, a user inputs the details of a task related to a transaction of an item and image data of the item to terminal device 10. Terminal device 10 transmits the details of the task and image data input by the user to management server 20. Management server 20 performs image processing on the image data according to the details of the task, and executes the procedures required for the task using the image-processed data. This enables a user to appropriately and easily perform tasks required for a commercial transaction of an item.

[0249] (2-6. Modifications) A modified example of the first embodiment will be described below with reference to Fig. 23 and Fig. 24. In the modified example, the control unit 204 of the management server 20 further includes an operation estimation module 2035. The operation estimation module 2035 estimates and presents candidate operations for an item based on image data input by the user. The operation estimation module 2035 executes, for example, an AI / ML search process to select operations corresponding to images similar to the image data input by the user, and presents the operations as candidate operations for the item.

[0250] The processing of the task estimation module 2035 in the modified example will be described with reference to Fig. 24. As an example, the task estimation module 2035 may acquire information on the attributes of the user (hereinafter also referred to as user attributes) together with the image data input by the user. The information on the user attributes may be, for example, information on whether the user is a member or a non-member. Furthermore, a user who is a member may include a member who utilizes (i.e., purchases and uses) an item and a member who provides (i.e., offers and sells) an item.

[0251] As shown in the example of FIG. 24, when the user attribute is a member who uses an item, the work estimation module 2035 compares image data input by the user by holding the terminal device 10 over the terminal device 10 or the like with image data of the item registered in the product information database 282. When the similarity between the item of the input image data and the item being used is high, the work estimation module 2035 displays items such as "return request", "assembly and disassembly instructions", "repair estimate", "similar product search", and "other" on the display 132 of the terminal device 10 to present them as work candidates. Here, in order to determine the similarity between the input image data and the image data registered in the product information database 282, an AI / ML search process that has learned using teacher data may be used. More specifically, for example, image data used for each history of work may be stored, and a learned model may be generated by machine learning for the image data and the work content. Then, the similarity between newly accepted image data and the history of image data on which each work was performed may be determined using the learned model, and a work with a high similarity may be presented as a work candidate.

[0252] Furthermore, for example, when the user attribute is a member who provides goods, image data input by the user by holding the terminal device 10 over the device, etc., is compared with image data of goods registered in the product information database 282. When there is a high degree of similarity between the goods in the input image data and the goods that the user is applying to use, the work estimation module 2035 displays items such as "Shipping", "Shipping Fee Estimate", "Repair Estimate", "Search for Similar Products", and "Other" on the display 132 of the terminal device 10 to present them as work candidates.

[0253] Also, for example, when the user attribute is non-member, image data input by the user by holding the terminal device 10 over the terminal device 10 or the like is compared with image data of the item registered in the product information database 282. When the similarity between the item of the input image data and the item registered in the product information database 282 is high, the task estimation module 2035 displays items such as "member registration", "login", "contact provider", and "other" on the display 132 of the terminal device 10 to present them as task candidates. Note that two or more attributes (for example, attributes as a member who uses an item and attributes as a member who provides an item) may be registered for one user. With such a configuration, the candidates of tasks to be selected by the user are narrowed down, so that it is possible to reduce the user's effort in selection.

[0254] <3. Third embodiment> Hereinafter, the system 1 according to the third embodiment of the present disclosure will be described with reference to FIGS.

[0255] (3-1. Overview of System 1) The system 1 in the third embodiment differs from the first embodiment in that the system 1 handles transactions relating to the repair of articles.

[0256] 25, the purchaser C operates the terminal device 10 to take a picture of the item P that he / she wishes to repair, and transmits the details of the necessary work and the image data to the management server 20. The management server 20 processes the received image data and registers an order requesting a quote for the item P. The management server 20 further estimates an estimated price and notifies the terminal device 10 of the purchaser C.

[0257] (3-2. Content of work) The content of the work in the second embodiment will be described with reference to Fig. 26. As shown in Fig. 26, in the task of requesting a repair estimate, a user (a person requesting repair) inputs image data of the exterior of a product into the terminal device 10. The user also inputs a request for an estimate as a task to be performed regarding a transaction. The terminal device 10 transmits the image data input by the user and the content of the work to the management server 20.

[0258] The image processing module 2033 of the control unit 204 of the management server 20 executes, for example, AI / ML search processing. The procedure execution module 2034 identifies the estimated cost of similar work based on the image data entered by the user and the searched image data, and calculates an estimated amount.

[0259] As described above, in the third embodiment, the technical ideas of the present disclosure can be applied to a system 1 that handles transactions related to the repair of goods, making it possible to appropriately and simply carry out the tasks necessary for commercial transactions of goods.

[0260] <4. Other embodiments> Although the embodiment and the modified examples of the present invention have been described above, application of the present disclosure is not limited to the above-mentioned contents.

[0261] For example, in the above embodiment, the Fingerprint of Things processing is performed as image processing in the unique information registration operation (see 2-4-2.) and the receiving operation (see 2-4-5.), but the present invention is not limited to this specification. For example, a label containing identification data such as a QR code (registered trademark) may be attached to an item, and the procedure required for the operation may be performed using image data of the item obtained by photographing the attached part.

[0262] In the above embodiment, some of the assumed processes are described as image processing of the image data of the item, but the present invention is not limited to this embodiment. For example, the present invention may be configured to use data processing technologies such as edge computing and block chains, or to perform image processing only on a terminal device without using a remote server. In addition, the image processing of the image data may be performed by an image processing module of another computer, and the image data may be handed over to the image processing module and the data after image processing may be received.

[0263] In the above embodiment, the terminal device 10 has been described as a terminal having a display 132 such as a smartphone, but the present invention is not limited to this example. The terminal device 10 may be realized by installing software on a home PC or the like, and may be configured to accept operations such as a mouse.

[0264] In addition, sharing of specific information via communication lines is mainly performed via a WAN such as the Internet, but between information processing devices, communication may also be performed only via short-range wireless communication such as Bluetooth (registered trademark) and infrared communication without using a WAN.

[0265] In addition, in the above embodiment, an example of a form in which each function is provided by the terminal device 10 or the management server 20 is described, but this is not limited to this form, and some functions may be provided by the terminal device 10, the management server 20, or both the terminal device 10 and the management server 20 in a form different from the above embodiment.

[0266] Moreover, each step described in the above embodiment as being executed by the terminal device 10 may also be executed by the management server 20 .

[0267] Furthermore, the present invention may be realized as a computer-readable non-transitory recording medium storing the above-mentioned program.

[0268] Various embodiments of the present invention have been described above, but these are presented as examples and are not intended to limit the scope of the invention. The novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. The embodiments and modifications thereof are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0269] 1: system, 10: terminal device, 13: input device, 14: output device, 15: memory, 16: storage, 17: processor, 20: management server, 25: memory, 26: storage, 27: processor, 30: external server, 40: network, 51: task selection icon, 52: image selection icon, 53: confirmation screen, 54: notification screen, 60: selection dial, 60a: center part, 60b: ring part, 61: candidate, 61a: user icon, 61b: radio button, 63: collaboration service management selection icon, 64: collaboration function management management selection icon, 65: add button, 66: linked service icon, 67: recording icon, 68: slide button, 100: commercial transaction management system, 111: antenna, 121: wireless communication unit, 130: operation reception unit, 131: device, 132: display, 140: audio processing unit, 141: microphone, 142: speaker, 150: camera, 160: sensor, 170: memory unit, 180: control unit, 201: communication unit, 202: memory unit, 203: input / output unit, 204: control unit, 281: user information database, 282: product Information database, 283: order information database, 301: communication unit, 302: memory unit, 303: input / output unit, 304: control unit, 1701: first action database, 1702: second action database, 1703: display action database, 1704: action linkage database, 1801: input operation reception unit, 1802: transmission / reception unit, 1803: data extraction unit, 1804: output control unit, 1805: data management unit, 2021: service database, 2022: service function database, 2023: Service action database, 2024: user database, 2025: provided function database, 2026: action database, 2031: receiving control module, 2032: sending control module, 2033: image processing module, 2034: procedure execution module, 2035: work estimation module, 2041: receiving control module, 2042: sending control module, 2043: data management module, 3041: receiving control module, 3042: sending control module, 3043: function execution module.

Claims

1. A system having at least one computer with a memory and a processor, The system includes a terminal device; The processor, receiving a first action for the terminal device for generating or selecting resource data to be used for a predetermined function provided in one or more services; presenting candidates for a second action for the terminal device for executing a function associated with the first action based on information pre-stored in the memory.

2. The system described in claim 1, which identifies functions associated with the user based on user information stored in the memory.

3. The system described in claim 1, which identifies candidates for the second action based on an association between resource data and functions stored in the memory.

4. The display of the first action and the second action candidates includes: The system according to claim 2 , wherein the learning models are displayed in an order based on weightings set based on at least one of the user information and the data type of the resource data.

5. storing either one of the processing to be performed by the first action or the processing to be performed by the second action; 2. The system according to claim 1, further comprising the step of receiving an instruction from an operator to repeat the other process.

6. The first action is an operation for taking or selecting an image of an item or identification information, The system of claim 1 , wherein the resource data is the captured or selected image.

7. The second action is: 3D data generation using the resource data, and generation of new files including image processing or synthesis processing; An information acquisition process including detecting a fingerprint of a thing from the resource data, detecting a defective part in the item, searching for a similar item, or acquiring assembly / disassembly instruction information for the item; and A status change process within a service, such as registration of a Fingerprint of Things in the resource data, delivery of the item, creation of an estimate for the item, or payment for the item; The system of claim 6 , further comprising an operation for performing at least one of the following:

8. The system according to claim 1 , wherein the elements displayed for selection of the first action are formed in a ring shape and configured to be rotatable.

9. The system according to claim 1 , wherein an element displayed for executing the first action or the second action is configured to be movable based on an operation by an operator.

10. A method for causing a computer having a memory and a processor to execute a process, comprising the steps of: The method further comprises: receiving a first action for the computer to generate or select resource data to be used for a predetermined function provided in one or more services; presenting suggestions of second actions to the computer for performing a function associated with the first action based on information pre-stored in the memory.

11. An information processing device including a memory and a processor, receiving a first action for the information processing device for generating or selecting resource data to be used for a predetermined function provided in one or more services; presenting candidates for a second action for the information processing device to execute a function associated with the first action based on information pre-stored in the memory.

12. A program for causing an information processing device having a memory and a processor to execute a process, receiving a first action for the information processing device for generating or selecting resource data to be used for a predetermined function provided in one or more services; presenting candidates for a second action for the information processing device to execute a function associated with the first action based on information pre-stored in the memory.

Images