Information processing system

Abstract

To provide an information processing system that generates the movement trajectory of an object like a person within a region where a range is defined in advance like a store.SOLUTION: An information processing system generates the movement trajectory of an object within a defined region that has a range defined in advance. This information processing system includes: a pre-learning model generating unit that generates a pre-learning model by machine learning using movement trajectory data which represents, by alphabetic information, the position of the object within the predefined region; and an execution processing unit that outputs the movement trajectory data using the pre-learning model.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing system that generates a movement trajectory of an object such as a person in an area with a predetermined range (hereinafter referred to as a "defined area") such as a store. [Background technology]

[0002] It is important to analyze the traffic lines, which are the trajectories of people's movements, in a specified area such as a store. For example, in the case of a store, how to lay out and arrange display shelves and how to arrange products are issues that directly affect sales, and are important when starting up a store. Also, for example, if the specified area is an event venue, where to place exhibition booths in order to move people smoothly is important from the perspective of preventing accidents and trouble.

[0003] For this reason, traditionally, people's movement tracks, or traffic lines, have been analyzed by, for example, taking photographs with cameras or installing sensors in various places.

[0004] Examples of systems that perform such analysis of traffic lines are disclosed in Patent Documents 1 and 2 below.

[0005] In addition, as shown in Non-Patent Document 1 and Patent Document 2 below, there are systems that estimate the movement of people within a store using a method called an agent-based model. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2017-174135 A [Patent Document 2] JP 2016-177583 A [Non-patent literature]

[0007] [Non-Patent Document 1] Toshiki Fujino and 6 others, "How do customers move in supermarkets? - What we can learn from customer movement analysis and agent simulation -", Society of Instrument and Control Engineers, 5th Social Systems Division Research Meeting Materials Vol. 5, p. 57-68, [online], Internet<URL:https: / / mas.kke.co.jp / wp-content / uploads / 2020 / 01 / thesis.pdf> [Non-Patent Document 2] Fujitsu Laboratories Ltd., Waseda University, "Development of technology to quickly discover causes of congestion through human behavior simulation," [online], Internet<URL:https: / / pr.fujitsu.com / jp / news / 2018 / 12 / 7.html> Summary of the Invention [Problem to be solved by the invention]

[0008] The systems in Patent Documents 1 and 2 take pictures of people in a store with a camera or detect them with a sensor, plot the pictures in a time series, and generate movement trajectories as traffic lines. Such systems are useful in that they can be used to analyze traffic lines within a store.

[0009] However, in order to actually detect the movement trajectories of people in a store by installing sensors in the store and fully detecting the movement lines of many people, it can be very expensive, costing up to tens of millions of yen. On the other hand, if you want to detect the movement lines of a small number of people, it can be done at a lower cost. In some cases, the movement lines can be generated by a person visually checking them.

[0010] Therefore, if the movement lines of many people can be generated (estimated) based on the movement lines of a small number of people, it will lead to a reduction in the overall cost burden. Also, if movement lines with fine time resolution (high accuracy) can be generated (estimated) by interpolating people's movement trajectories from movement lines with coarse time resolution (low accuracy), it will also lead to a reduction in the overall cost burden. Therefore, there is a demand to generate (estimate) the movement lines of many people or movement lines with fine time resolution from the movement lines of a small number of people or movement lines with coarse time resolution.

[0011] In addition, in order to generate a person's movement line, it is necessary to photograph the moving person with a camera and detect them with a sensor, so the movement line is obtained based on the actual movement of the person. Therefore, it is not easy to generate a movement line in a situation where the person is not actually moving.

[0012] Therefore, as shown in the above-mentioned Non-Patent Document 1 and Non-Patent Document 2, there are systems that estimate people's movements in a store using a method called an agent-based model. However, when using an agent-based model, it is necessary to model in advance how people move, and the modeling itself is a burden. In addition, since people move freely in a store, they may behave in unexpected ways, but such movements cannot be estimated, and there are situations in which the accuracy cannot be said to be high.

[0013] Therefore, there is a need for a method to generate (estimate) movement paths with less load and greater accuracy than agent-based models.

[0014] In view of the above-mentioned problems, the present inventor has invented an information processing system that generates a movement trajectory of an object such as a person in a specified area, such as a store, whose scope is determined in advance.

[0015] A first invention is an information processing system that generates a movement trajectory of an object in a specified area, which is an area with a predetermined range, the information processing system having a pre-learning model generation processing unit that performs machine learning using movement trajectory data indicating the position of the object in the specified area as text information to generate a pre-learning model, and an execution processing unit that outputs the movement trajectory data using the pre-learning model.

[0016] A second invention is an information processing system that generates a movement trajectory of an object in a specified area, which is an area with a predetermined range, the information processing system having a pre-learning model that has been machine-learned using first movement trajectory data indicating the position of the object in the specified area as character information, a transfer learning model generation processing unit that generates a transfer learning model by performing transfer learning using second movement trajectory data indicated as character information, and an execution processing unit that outputs the movement trajectory data using the transfer learning model.

[0017] By using these inventions, it becomes possible to generate the movement trajectory of an object such as a person in a predetermined area such as a store. Also, since there is no need to model the movement of the object in advance as in the agent-based model, it is possible to generate a movement trajectory with high accuracy and with less load.

[0018] In the above-mentioned invention, the information processing system can be configured as an information processing system having a pre-learning model generation processing unit that performs machine learning using first movement trajectory data indicating the position of an object in a first specified area using character information to generate a pre-learning model, the transfer learning model generation processing unit generates the transfer learning model by performing transfer learning using second movement trajectory data indicating the position of an object in a second specified area using character information and the pre-learning model, and the execution processing unit uses the transfer learning model to output movement trajectory data indicating the position of an object in the second specified area using character information.

[0019] In the second invention, as in the present invention, transfer learning is performed using a pre-learning model generated using the first movement trajectory data in the first specified area and the second movement trajectory data in the second specified area, so that the movement trajectory data in the second area can be output. This makes it possible to generate, for example, the movement trajectories of customers in a newly opened store.

[0020] In the above-mentioned invention, the information processing system can be configured as an information processing system having a pre-learning model generation processing unit that performs machine learning using first movement trajectory data indicating the position of the target in the specified area using character information to generate a pre-learning model, the transfer learning model generation processing unit generates the transfer learning model by performing transfer learning using second movement trajectory data indicating the position of the target in the specified area using character information and the pre-learning model, and the execution processing unit uses the transfer learning model to output movement trajectory data indicating the position of the target in the specified area using character information.

[0021] In the second invention, the movement trajectory data may be output by performing transfer learning using a pre-learning model generated using the first movement trajectory data in a specified area as in the present invention and the second movement trajectory data. For example, when a pre-learning model is generated based on the movement trajectories of customers at a certain store, it may be desired to generate the movement trajectories of customers when there is an event such as a fireworks display. Since events such as fireworks displays are not always held, there are not many samples of the movement trajectories of customers, and there may be only a few samples. In such a case, it is not easy to perform pre-learning using the movement trajectories themselves. Therefore, as in the present invention, pre-learning is performed using the movement trajectories at normal times, and transfer learning is performed using a small amount of movement trajectory data when an event such as a fireworks display occurs, so that a lot of movement trajectory data when an event such as a fireworks display occurs can be generated.

[0022] In this way, even if there are only a small number of trajectory samples, a large number of trajectories can be generated with high accuracy.

[0023] In the above-mentioned invention, the information processing system can be configured as an information processing system having a data conversion processing unit that converts movement line data including the position of an object in a specified area into movement trajectory data indicated by text information.

[0024] The position of the target in the specified area is usually generated by movement line data. The movement line data is generally coordinate information, and is not easy to process as it is. Therefore, by converting it into movement trajectory data represented by character information as in the present invention, machine learning can be performed.

[0025] In the above-mentioned invention, the data conversion processing unit can be configured as an information processing system in which the data conversion processing unit converts the movement trajectory data output by the execution processing unit into movement line data, and the execution processing unit displays the movement line data converted by the data conversion processing unit.

[0026] Since the movement trajectory data output by the execution processing unit is text information, it is not easy for humans to understand the movement trajectory of the target as it is. Therefore, as in the present invention, by converting (reverse converting) the movement trajectory data of the output text information into movement line data, it is possible to visualize the movement line.

[0027] In the above-mentioned invention, the movement trajectory data can be configured as an information processing system in which the specified area is divided into a plurality of areas in a multi-layered manner and character information is added to each divided area, thereby indicating the position of the target in the specified area with a plurality of character information.

[0028] With the configuration of the present invention, the position of an object in a specified area can be indicated with a small amount of text information.

[0029] In the above-mentioned invention, the movement trajectory data can be configured as an information processing system including character information indicating the position of the object and character information indicating a movement time interval.

[0030] The movement trajectory data can be displayed in various ways, and it is sufficient if it contains at least text information indicating the position of the target. If the position of the target is only text information, the position must be acquired at regular intervals. In that case, the amount of data of the movement trajectory data increases. Therefore, the amount of data can be reduced by acquiring the position every time the target moves, and including the position and the time interval until the movement as text information in the movement trajectory data.

[0031] In the above-mentioned invention, the movement trajectory data can be configured as an information processing system including character information indicating the position of the target and character information indicating an action.

[0032] The movement trajectory data may contain text information indicating the target's actions in addition to the target's position. In this case, the target's actions at that position can also be processed, so that pre-learning and transfer learning of the movement trajectory including the target's actions at that position, such as picking up a product or looking at a product, can be performed to estimate the target's actions.

[0033] In the above-mentioned invention, the movement trajectory data can be configured as an information processing system including character information indicating the position of the object and character information indicating the environment.

[0034] The movement trajectory data may include text information indicating the environment in addition to the target location. In this case, a more detailed movement trajectory can be generated by taking into account external factors such as the weather, temperature, date, time period, target attributes, the presence or absence and type of events, and the environment that may affect the target movement trajectory.

[0035] In the above-mentioned invention, the machine learning can be configured as an information processing system using a deep learning model or a probabilistic model.

[0036] In the present invention, since character information is used as movement trajectory data, it is preferable to use a deep learning model or a probabilistic model for machine learning.

[0037] The first invention can be realized by loading and executing the information processing program of the present invention into a computer. That is, the information processing program causes the computer to function as a pre-learning model generation processing unit that performs machine learning using movement trajectory data indicating the position of an object in a specified area by character information to generate a pre-learning model, and an execution processing unit that uses the pre-learning model to output the movement trajectory data.

[0038] The second invention can be realized by loading and executing the information processing program of the present invention into a computer. That is, the information processing program causes the computer to function as a transfer learning model generation processing unit that generates a transfer learning model by performing transfer learning using a pre-learning model that has been machine-learned using first movement trajectory data indicating the position of a target in a specified area using character information and second movement trajectory data indicated using character information, and an execution processing unit that outputs movement trajectory data using the transfer learning model. Effect of the Invention

[0039] By using the information processing system of the present invention, it is possible to generate a movement trajectory of an object such as a person in a specified area, such as a store, whose range is determined in advance. [Brief description of the drawings]

[0040] [Figure 1] 1 is a block diagram showing an example of a system configuration of an information processing system according to the present invention; [Diagram 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer that realizes the information processing system of the present invention. [Diagram 3] 11 is a flowchart illustrating an example of a pre-learning model generation process of the information processing system of the present invention. [Figure 4] 4 is a flowchart showing an example of an execution process of the information processing system of the present invention. [Diagram 5] FIG. 11 is a plan view of a store as an example of a defined area. [Figure 6] FIG. 13 is a diagram showing a state in which a specified region is divided into each region. [Figure 7] 6 is a diagram showing an example of target movement line data in the specified area of ​​FIG. 5. FIG. [Figure 8] 8 is a diagram showing an example of a state in which the flow line data in FIG. 7 is converted into movement trajectory data made up of character information. FIG. [Figure 9] FIG. 10 is a diagram illustrating an example of a state in which movement trajectory data is output using a pre-learning model and the movement line data is displayed. [Figure 10] FIG. 11 is a block diagram illustrating an example of a system configuration of an information processing system according to a second embodiment. [Figure 11] FIG. 13 is a diagram illustrating an example of target flow line data in the second embodiment. [Figure 12] 12 is a diagram showing an example of a state in which the flow line data in FIG. 11 is converted into movement trajectory data made up of character information. FIG. [Figure 13] FIG. 11 is a diagram illustrating an example of a state in which movement trajectory data is output using a transfer learning model in the second embodiment and the movement line data is displayed. [Figure 14] 1 is a flowchart illustrating an example of a transfer learning model generation process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0041] FIG. 1 is a block diagram showing an example of the overall processing function of an information processing system 1 according to the present invention.

[0042] The information processing system 1 of the present invention is realized by a computer. An example of the hardware configuration of a computer used in the information processing system 1 is shown in schematic form in Fig. 2. The computer preferably has a calculation device 70 such as a CPU that executes calculation processing of a program, a storage device 71 such as a RAM or a hard disk that stores information, a display device 72 such as a display that displays information, an input device 73 such as a keyboard or a mouse that can input information, and a communication device 74 that transmits and receives the processing results of the calculation device 70 and information stored in the storage device 71 via a network such as the Internet or a LAN, but is not limited thereto, and some or all of the display device 72, the input device 73, and the communication device 74 may not be provided.

[0043] In the case where the computer is equipped with a touch panel display, the display device 72 and the input device 73 may be integrally configured. Touch panel displays are often used in portable communication terminals such as tablet computers and smartphones, but are not limited to these.

[0044] The touch panel display is a device that integrates the functions of the display device 72 and the input device 73 in that input can be made directly on the display by a predetermined input device (such as a touch panel pen) or a finger.

[0045] The information processing system 1 of the present invention may be realized by one computer or may be composed of multiple computers. In addition, a part or the whole of the information processing system 1 may be realized by cloud computing. Furthermore, the information processing system 1 may be incorporated in another computer system or device and may be a part of it. In addition, the processing of the information processing system 1 of the present invention may be realized as a control circuit.

[0046] The functions of the various means in the present invention are only logically distinct, and may be physically or practically the same area. The order of the processes in the various means of the present invention may be changed as appropriate. In addition, some of the processes may be omitted.

[0047] The information processing system 1 of the present invention converts data on flow lines showing the movement of objects such as people in a specified area with a predetermined range into movement trajectory data of character information, and uses the movement trajectory data to perform machine learning using deep learning models including large-scale language models (LLMs) and conditional probability models such as Markov chains (modeling the probability that the next character will appear under conditions in which a certain character has appeared).Then, using the learning model, the movement trajectory data of the object is output and converted into movement line data, thereby displaying the movement line of the object.

[0048] The specified area refers to a two-dimensional or three-dimensional area with a predetermined range, such as inside a store, inside a structure such as a building, inside an event venue, etc. An example of the specified area is a closed spatial area (closed area) such as the space inside a building, but it can be any area that can be virtually divided into a predetermined range, whether indoors or outdoors, as described below. In the following description of this specification, the case of inside a store will be described, but the present invention is not limited to this.

[0049] The movement trajectory of an object in a specified area can be represented by text information, for example, as follows.

[0050] FIG. 5 is a plan view of a store as an example of a specified area. This specified area is divided as shown in FIG. 6. For example, first, the entire area of ​​the store, which is the specified area, is divided into four areas, and each of the divided areas is divided into four areas. This division of the area is repeated several times until the area is large enough to analyze the movement trajectory of the target person. FIG. 6 shows a state in which the area has been divided six times, but the number of divisions may be any number. Note that each area does not need to be divided into four areas (2×2), but may be divided into m×n areas. m=n may be acceptable. Also, each area does not need to be divided into the same number of areas. For example, it may be divided into six areas at first, and each of the areas may be divided into four areas. Furthermore, each area does not need to be divided into quadrangles, and may be divided into areas of any two-dimensional shape including polygons such as triangles, pentagons, and hexagons.

[0051] In addition to dividing the area in two dimensions, the area may be divided in three-dimensional space. For example, when the specified area is the three-dimensional space of a store and the store has a hierarchical structure such as two or three stories, or when it is necessary to generate target movement line data and movement trajectory data in the height direction such as above and below a display shelf, the area (area in three-dimensional space) may be divided into any solid shape including polyhedrons such as cubes and rectangular parallelepipeds.

[0052] In this specification, a case where the specified area is divided into rectangles will be described, but similar processing can be performed even if the area is divided into shapes other than rectangles, or if the specified area is divided into a solid in a three-dimensional space.

[0053] As shown in FIG. 6, identification information is given to each divided area. FIG. 6 shows a state in which the entire area of ​​the specified area is divided into four areas, a, b, c, and d, area a is further divided into four areas, e, f, g, and h, and area e is further divided into four areas, i, j, k, and l. Of course, area b is divided into area d, and area f is divided into area h in the same way. This division of the area is performed six times, and identification information is given to each area. Note that, as for the identification information, if the layers of division (number of divisions) are different, the same identification information can be used as the identification information for the areas. For example, if the area is divided into four areas, a, b, c, and d, in the first division, the same identification information as a, b, c, and d can be used in the second division to divide area a. In this way, by expressing the identification information of the areas of each layer corresponding to a certain position according to a predetermined rule, the position can be indicated by multiple character information.

[0054] As shown in FIG. 6, when the region is divided six times, any position (region) in the specified region can be represented by six characters. 1 In the specified area, the position (area) where the object is located is P 1 =ζ 1 ζ 2 ζ 3 ζ 4 ζ 5 ζ 6 This can be shown as:

[0055] By listing this text information in chronological order, the movement lines in the target specified area can be expressed as text information. An example of this is shown in Figures 7 and 8. Figure 7 is an example of the movement lines of people in a store, and Figure 8 is an example of the movement lines in Figure 7 converted into movement trajectory data using text information.

[0056] In addition, a specified symbol (character information) such as a space or a comma may be inserted to separate the character information between time points, and if the number of characters at a certain time point is fixed, the character information at each time point can be determined for each number of characters.

[0057] The following description will be given of a human as the object moving within the specified area, but this is not limited to a human, and may include living things such as animals and insects, as well as autonomously moving devices, and any other tangible object that can move autonomously.

[0058] The information processing system 1 includes a data conversion processing unit 10, a data reception processing unit 11, a pre-learning model generation processing unit 12, a pre-learning model storage unit 13, and an execution processing unit 14.

[0059] The data conversion processing unit 10 converts the movement line data of the object moving in the specified area into movement trajectory data consisting of character information. It also converts (reverse converts) the movement trajectory data consisting of character information into movement line data. The movement line data is data showing the state of the movement of the object in the specified area by connecting the movement of the object with a line, and may be in a state that can be illustrated as shown in FIG. 7, or may be coordinate information arranged in chronological order.

[0060] Conversion from flow line data to movement trajectory data and conversion from movement trajectory data to flow line data (reverse conversion) is performed by, for example, allocating coordinate information capable of identifying the range of each area, such as four points in each area or two points in the case of a square area, to each area obtained by dividing a specified area. Then, conversion from flow line data to movement trajectory data can be performed by comparing the coordinate information at each time point in the flow line with the coordinate information of each area.

[0061] Furthermore, the movement trajectory data can be converted into flow line data by connecting the center points of the areas in the movement trajectory data represented by text information with a line.

[0062] Conversion from flow line data to movement trajectory data and conversion (inverse conversion) from movement trajectory data to flow line data can be performed using various methods, and the above is just one example, and other methods may also be used.

[0063] An example of the movement trajectory data is movement trajectory data that indicates the position information of an object moving in a specified area at regular intervals. In this case, the movement trajectory data is Movement trajectory data = {[text information indicating location]} It should be noted that the braces are used as a symbol to indicate repetition.

[0064] In this case, the movement trajectory data is a character string in which character information indicating a position is repeated at regular intervals.

[0065] In the case of the above-mentioned movement trajectory data, the amount of data increases because the position information at regular intervals is converted into text information. Therefore, the amount of movement trajectory data can be reduced by measuring the position information every time an object moves within a specified area.

[0066] That is, as another example of the movement trajectory data, the movement trajectory data may be configured to include character information indicating position information and characters indicating a movement time interval. The characters indicating the movement time interval are the time interval from one point to the next point, and are expressed as a predetermined character. For example, 0 to 2 seconds may be expressed as 2 (character 2), 2 to 4 seconds as 4 (character 4), 4 to 6 seconds as 6 (character 6), 6 to 8 seconds as 8 (character 8), 8 to 10 seconds as A, and 10 to 12 seconds as B, and other predetermined time intervals may be expressed as character information. Also, the character information indicating the movement time interval may be expressed by two or more characters instead of one character.

[0067] In this case, the movement trajectory data is, for example, Movement trajectory data = {[Text information indicating position][Text information indicating movement time interval]} It should be noted that the braces are used as a symbol to indicate repetition.

[0068] In this case, the amount of data required for the movement trajectory data is reduced because the position information is recorded according to the movement. This reduces the processing load of the pre-learning model generation processing unit 12, such as pre-learning.

[0069] In this case, the movement trajectory data may be expressed as a character string according to some rule, with the character information indicating the position and the character information indicating the movement time interval as a pair. The order of the character information indicating the position and the character information indicating the movement time interval may be reversed. Furthermore, the character information indicating the position and the character information indicating the movement time interval do not have to be adjacent to each other.

[0070] Furthermore, each of the above-mentioned examples of the movement trajectory data may further include character information indicating an action as the movement trajectory data. That is, the movement trajectory data may be configured to include character information indicating position information, character information indicating an action, and, in some cases, character information indicating a movement time interval.

[0071] The character information indicating an action is associated with the action in advance, such as "viewing a product" as A, "picking up a product" as B, and "putting a product in a basket" as C, and indicates the action that the subject performed at that location as character information. The character information indicating an action is not limited to the above, and can be set arbitrarily according to the specified area, and may be expressed with two or more characters instead of one character.

[0072] The text information indicating the action can be identified, for example, from a security camera installed in a specified area. In addition, if the flow line of a customer can be associated with a purchase in a POS system, the action of the customer at the display position of the purchased product can be identified as an action such as "putting the product in a basket."

[0073] In this case, the movement trajectory data is, for example, Movement trajectory data = {[Text information indicating location][Text information indicating action][Text information indicating movement time interval]} It should be noted that the braces are used as a symbol to indicate repetition.

[0074] In this case, the movement trajectory data records the target's actions, so it is possible to infer the actions the target takes in a certain location, enabling more detailed inferences to be made, including where the product was purchased.

[0075] In this case, the movement trajectory data may be expressed as a string of characters according to some rule, in which text information indicating a position in a specified area, text information indicating an action, and possibly text information indicating a movement time interval are paired. The order of these may be determined arbitrarily. The text information indicating a position, the text information indicating a movement time interval, and the text information indicating an action do not have to be adjacent to each other.

[0076] In addition, in the case of this movement trajectory data, character information indicating an action is added to the movement trajectory data of the second example. However, if character information indicating an action is added to the movement trajectory data of the first example, character information indicating the movement time interval is not necessary.

[0077] Furthermore, as another example of the movement trajectory data, information indicating the environment may be provided in the movement trajectory data. The information indicating the environment is expressed as character information regarding various information related to the environment that may affect the generation of the movement line, such as attributes of the object moving through the specified area, such as gender and age, weather, temperature, date, day of the week, time or time period, the presence or absence or type of an event, external factors such as the layout pattern of the display shelves, etc.

[0078] For example, for gender, M is for male, W is for female, and U is for unknown; for weather, S is for sunny, C is for cloudy, and R is for rain; and for dates, text information indicating the date is included as information indicating the environment.

[0079] The information indicating the environment may be inputted by a predetermined method and may be included as part of the character string of the movement trajectory data.

[0080] In this case, the movement trajectory data is, for example, Movement trajectory data = [text information indicating the environment] {[text information indicating the position] [text information indicating the action] [text information indicating the movement time interval]} It should be noted that the braces are used as a symbol to indicate repetition.

[0081] In this case, the movement trajectory data also contains information indicating the environment, making it possible to make more detailed estimations including the attributes of the target and external factors such as the weather.

[0082] In this case, the movement trajectory data may include at least information indicating the environment and character information indicating the position, and may include character information indicating the action and character information indicating the movement time interval as necessary.

[0083] Note that this data representation is merely an example, and it is sufficient that the data is expressed as a character string by being repeated according to some rule, and the order of the data may be determined arbitrarily.

[0084] As for the movement trajectory data used in the information processing system 1 of the present invention, as described above, various patterns can be used and processing can be performed in the same manner.

[0085] The data reception processing unit 11 receives input of movement trajectory data that expresses the movement trajectory of an object such as a person moving within a specified area using text information. For example, the data reception processing unit 11 receives input of movement trajectory data as shown in Fig. 8, and executes machine learning processing in the pre-learning model generation processing unit 12 described later.

[0086] The pre-learning model generation processing unit 12 receives input of multiple trajectory data in a specified area for pre-learning from the data reception processing unit 11, and executes machine learning (pre-learning) processing of the trajectory data using a deep learning model such as a large-scale language model (LLM) or a conditional probability model. In this case, the trajectory data is input to a learning model in which the weighting coefficient between neurons in each layer of a neural network consisting of many intermediate layers is optimized, and the output value is output. In addition, as the learning model, a model in which various trajectory data in a specified area are given as learning data can be used. In addition, as the large-scale language model (LLM), various models such as a GPT system that outputs future trajectory data and a BERT system that interpolates trajectory data can be used.

[0087] The pre-learning model generation processing unit 12 stores the pre-learning model (first learning model) generated in the pre-learning process in the pre-learning model storage unit 13.

[0088] The pre-learning model storage unit 13 stores the pre-learning model generated by the pre-learning model generation processing unit 12.

[0089] The execution processing unit 14 uses the pre-learning model stored in the pre-learning model storage unit 13 to output movement trajectory data. That is, by giving a predetermined instruction to the pre-learning model, it outputs movement trajectory data consisting of character information. Then, the output movement trajectory data consisting of character information is passed to the data conversion processing unit 10, which converts (reverse converts) the movement trajectory data into movement line data. By accepting and displaying this movement line data from the data conversion processing unit 10, it is possible to display the movement line of the object generated by the pre-learning model.

[0090] FIG. 9 shows an example of a state in which movement trajectory data is output by the execution processing unit 14 using the pre-learning model and the flow line data is displayed. EXAMPLES

[0091] An example of the processing of the information processing system 1 of the present invention will be described with reference to the flowcharts of Fig. 3 and Fig. 4. In the following embodiments, as described above, the specified area is the inside of a store, and the target is a person (a customer). The movement trajectory data in the form of a character string is measured at regular intervals, for example, every 5 seconds, and the measured data is converted into a character string in advance. The movement trajectory data may use location information such as GPS, may be location information detected by sensors installed in various places in the store, may be location information detected using a beacon, or may be location information photographed by various cameras installed in the store, and various other data may be used.

[0092] In addition, the information processing system 1 of the present invention is used to estimate and display the flow line data of new customers at a store A based on the flow line data of customers at the store A. Note that in the following description of each embodiment, the case where deep learning is used as the machine learning will be described.

[0093] First, the data conversion processing unit 10 converts the movement line data of customers visiting Store A into movement trajectory data consisting of text information (S100). For use in pre-learning, for example, several hundred to a thousand or more pieces of movement line data may be converted into movement trajectory data. The movement trajectory data to be converted at this time should be a necessary and sufficient number for generating a pre-learning model. As an example, the data conversion processing unit 10 converts the movement trajectory data of 1,000 or more customers. Note that 1,000 pieces is just an example, and the number of pieces may be more or less as long as it can be learned by the pre-learning model.

[0094] Then, the movement trajectory data converted by the data conversion processing unit 10 is received by the data reception processing unit 11 (S110). For example, the data reception processing unit 11 receives movement trajectory data consisting of 1000 or more pieces of character information.

[0095] The pre-learning model generation processing unit 12 performs machine learning using the movement trajectory data converted into text information of customers at Store A received from the data reception processing unit 11 (S120), generates a pre-learning model, and stores it in the pre-learning model memory unit 13 (S130).

[0096] By executing the above-mentioned processing, a pre-learning model (first learning model) for store A can be generated.

[0097] Next, the execution processing unit 14 performs execution processing using the pre-learning model (first learning model) thus generated.

[0098] The information processing system 1 receives an instruction to generate a flow line in store A (S200). For example, flow line data with a coarse time resolution (low accuracy) is received, and the data conversion processing unit 10 converts the flow line data with a coarse time resolution into movement trajectory data consisting of text information. The converted (coarse time resolution) movement trajectory data is input to the data reception processing unit 11, and the execution processing unit 14 uses the pre-learning model stored in the pre-learning model storage unit 13 to output movement trajectory data with a fine time resolution (high accuracy) (S210).

[0099] In addition, when generating trajectory data with fine time resolution from trajectory data with coarse time resolution, it is preferable to use a machine learning model that performs data interpolation, such as a BERT-based machine learning model.

[0100] The execution processing unit 14 converts (reverse converts) the generated movement trajectory data with fine time resolution into movement line data in the data conversion unit (S220), and displays the converted movement line data. By executing the above-mentioned process, it becomes possible to generate movement line data with fine time resolution (high accuracy) from movement line data with coarse time resolution (low accuracy) using the pre-learning model.

[0101] In addition, although the above describes a case where flow line data with a coarse time resolution is output as flow line data with a fine time resolution, the execution processing unit 14 may generate and output flow line data of a large number of people from the flow line data of a small number of people.

[0102] In this case, the execution processing unit 14 of the information processing system 1 receives character information indicating the initial position, and outputs character information indicating the position at the next time point using the pre-learning model stored in the pre-learning model storage unit 13. The output character information is then recursively input to the pre-learning model as an input value again. By connecting these in succession, the execution processing unit 14 outputs movement trajectory data using the pre-learning model, and generates movement line data.

[0103] For example, the information processing system 1 receives input of an instruction to generate a flow line in store A, the coordinates of the starting position of the flow line, and an instruction for the number of movement trajectories to be generated, for example, 100 (S200). Then, the data conversion processing unit 10 converts the coordinates of the starting position of the flow line into movement trajectory data (character information indicating the starting position) consisting of character information. Then, the execution processing unit 14 uses the pre-learning model stored in the pre-learning model storage unit 13 to output character information indicating the position at the next time point based on the input character information. By performing this process recursively, movement trajectory data consisting of character information from the start position to the end position is output (S210). This process is performed for the number of movement trajectory data to be generated, for example, 100, and is output.

[0104] In addition, when generating a large amount of movement trajectory data in this manner, it is preferable to use a machine learning model for generating the data, such as a GPT-based machine learning model.

[0105] The execution processing unit 14 converts (reverse converts) the generated large number of pieces of movement trajectory data, for example 100 pieces, into flow line data in the data conversion unit (S220), and displays the converted flow line data. By executing the above-mentioned processing, it becomes possible to use the flow line data of a small number of people to generate and output flow line data of many people by the pre-learning model. EXAMPLES

[0106] In the above embodiment, the case where the pre-trained model is used as is has been described. However, for example, when opening a new store, a transfer learning model that has undergone transfer learning on the pre-trained model may be used.

[0107] An example of the overall configuration of the information processing system 1 in this case is shown in FIG.

[0108] The information processing system 1 in this embodiment includes a transfer learning model generation processing unit 15 and a transfer learning model storage unit 16 in addition to the components in the first embodiment.

[0109] The transfer learning model generation processing unit 15 receives the input of the pre-learning model stored in the pre-learning model storage unit 13 and new movement trajectory data received by the data reception processing unit 11, executes machine learning (transfer learning) processing using the pre-learning model (first learning model), generates a transfer learning model (second learning model), and stores it in the transfer learning model storage unit 16. Note that transfer learning in this specification also includes fine tuning of the pre-learning model. In transfer learning, only new layers to be added to the pre-learning model are additionally learned, and in fine tuning, the entire model layer or a part of the layers in the pre-learning model are additionally learned. Various known methods can be used for transfer learning including fine tuning.

[0110] Since the transfer learning model generation processing unit 15 performs transfer learning using the pre-learning model stored in the pre-learning model storage unit 13, the transfer learning model generation processing unit 15 also performs machine learning (transfer learning) processing of movement trajectory data using a deep learning model such as a large-scale language model (LLM).

[0111] If the pre-learning model was generated based on the movement trajectory data of customers in store A, the data reception processing unit 11 receives input of a small number of movement trajectory data of customers in store B, and the transfer learning model generation processing unit 15 performs transfer learning to generate a transfer learning model. The movement trajectory data of customers in store B received at this time may be a small number of sample data of actual customers in store B, or may be virtual movement trajectory data created as samples. Then, the movement trajectory data of customers in store B is generated and output.

[0112] Fig. 11 shows an example of the flow line data of a customer in store B. Fig. 12 shows an example of movement trajectory data made up of character information converted from the flow line data in Fig. 11 by the data conversion processing unit 10.

[0113] In addition, the data reception processing unit 11 receives input of a pre-learning model pre-learned at store A and a small number of movement trajectory data of customers at store A under predetermined conditions, for example, conditions at the time when an event is held, and the transfer learning model generation processing unit 15 performs transfer learning to generate a transfer learning model. The movement trajectory data of customers under predetermined conditions in store A received at this time may be a small number of sample data of actual customers under predetermined conditions at store A, or may be virtual movement trajectory data created as a sample. Movement trajectory data under predetermined conditions at store A is generated and output.

[0114] The transfer learning model storage unit 16 stores the transfer learning model generated by the transfer learning model generation processing unit 15.

[0115] Moreover, the execution processing unit 14 in this embodiment uses the transfer learning model stored in the transfer learning model storage unit 16 to output movement trajectory data. That is, by giving a predetermined instruction to the transfer learning model, it outputs movement trajectory data consisting of character information. Then, the output movement trajectory data consisting of character information is passed to the data conversion processing unit 10, which converts (reverse converts) the movement trajectory data into movement line data. By accepting and displaying this movement line data from the data conversion processing unit 10, it is possible to display the movement line of the target generated by the transfer learning model.

[0116] FIG. 13 shows an example of a state in which movement trajectory data is output by the execution processing unit 14 using a transfer learning model and the movement line data is displayed.

[0117] Next, an example of the processing of the information processing system 1 in this embodiment will be described with reference to the flowcharts of Figures 3, 4, and 14. In the following description, a case will be described in which a small amount of movement trajectory data of a newly opened store B is transferred to a pre-learning model based on the movement trajectory data of store A to generate a traffic line at store B, but even in the case of transferring movement trajectory data under a predetermined condition at store A to generate a traffic line under a predetermined condition at store A as described above, the processing can be executed in the same manner, with only the difference being the type of movement trajectory data input and output in transfer learning.

[0118] The process of generating a pre-trained model (FIG. 3) is the same as that in the first embodiment, and therefore the description will be omitted.

[0119] The number of movement line data of customers coming to the newly opened store B, for example about 10 to 100, required for transfer learning is created on a drawing. The created movement line data is converted into movement trajectory data consisting of character information by the data conversion processing unit 10, and the character information is converted into character strings to generate movement trajectory data consisting of character strings (S300). That is, the data conversion processing unit 10 converts and generates the movement line data shown in FIG. 11 into the movement trajectory data shown in FIG. 12.

[0120] This allows for the generation of around 10 to 100 virtual movement trajectory data for Store B. This movement trajectory can be created appropriately by the person in charge of opening the store, based on experience, etc. It is preferable for the movement trajectory data for customers visiting Store B to be around 10 to 100, but it may be less or more than that.

[0121] Then, the movement trajectory data converted by the data conversion processing unit 10 (for example, the movement trajectory data shown in FIG. 12) is received by the data reception processing unit 11 (S310). For example, the data reception processing unit 11 receives movement trajectory data consisting of about 10 to 100 pieces of character information.

[0122] Then, the transfer learning model generation processing unit 15 performs transfer learning using the pre-learning model stored in the pre-learning model storage unit 13 and the movement trajectory data of Store B in the form of character string converted into text information received in S310 (S320), generates a transfer learning model, and stores it in the transfer learning model storage unit 16 (S330).

[0123] By performing the above-mentioned processing, a transfer learning model (second learning model) for store B can be generated.

[0124] Next, the execution processing unit 14 performs execution processing using the transfer learning model (second learning model) thus generated.

[0125] The information processing system 1 receives an instruction to generate a traffic line in store B (S200). That is, by receiving an operation to generate a traffic line in the execution processing unit 14, similarly to the first embodiment, the execution processing unit 14 uses the transfer learning model stored in the transfer learning model storage unit 16 to output a predetermined number of, for example, about 1000 pieces of character-based movement trajectory data (S210).

[0126] The execution processing unit 14 converts (reverse converts) the generated large number of movement trajectory data into flow line data in the data conversion unit (S220), and displays the converted flow line data. For example, the flow lines converted in S220 can be plotted and displayed on a floor plan of the store layout of Store B, thereby displaying the flow lines of Store B (S230). This is shown diagrammatically in FIG. 13.

[0127] By executing the above-mentioned processing, it is possible to generate and display traffic lines for the newly opened store B at the same level as store A by performing transfer learning using a small amount of movement trajectory data for store B. Based on the traffic lines generated here, it is possible to perform simulations of the layout of store shelves and the products to be displayed on the shelves when the new store B opens. EXAMPLES

[0128] Even if the movement trajectory data includes not only position information but also character information indicating a movement time interval, character information indicating an action, character information indicating an environment, etc., it is possible to execute processing similar to that of each of the above-mentioned embodiments.

[0129] The order of the processes in the information processing system 1 of the present invention may be changed as appropriate, and some of the processes may be omitted. [Industrial Applicability]

[0130] By using the information processing system 1 of the present invention, it is possible to generate a movement trajectory (traffic line) of a person in a specified area. [Explanation of symbols]

[0131] 1: Information processing system 10: Data conversion processing section 11: Data reception processing unit 12: Pre-training model generation processing unit 13: Pre-training model memory unit 14: Execution processing unit 15: Transfer learning model generation processing unit 16: Transfer learning model memory section 70: Arithmetic device 71:Storage device 72:Display device 73: Input device 74:Communication equipment

Claims

1. An information processing system that generates a movement trajectory of an object in a specified area, which is an area having a predetermined range, comprising: The information processing system includes: a pre-learning model generation processing unit that performs machine learning using movement trajectory data indicating the position of a target in a specified region by character information to generate a pre-learning model; an execution processing unit that outputs movement trajectory data using the pre-learning model; An information processing system comprising:

2. An information processing system that generates a movement trajectory of an object in a specified area, which is an area having a predetermined range, comprising: The information processing system includes: a transfer learning model generation processing unit that generates a transfer learning model by performing transfer learning using a pre-learning model that has been machine-learned using first movement trajectory data indicating the position of a target in a specified region using character information and second movement trajectory data indicated using character information; an execution processing unit that outputs movement trajectory data using the transfer learning model; An information processing system comprising:

3. The information processing system includes: a pre-learning model generation processing unit that performs machine learning using first movement trajectory data indicating a position of a target in a first specified area by character information to generate a pre-learning model, The transfer learning model generation processing unit includes: generating the transfer learning model by performing transfer learning using second movement trajectory data indicating the position of the target in a second specified area by character information and the pre-learning model; The execution processing unit: outputting movement trajectory data indicating a position of a target in the second specified area by text information using the transfer learning model; 3. The information processing system according to claim 2.

4. The information processing system includes: a pre-learning model generation processing unit that performs machine learning using first movement trajectory data indicating a position of a target in the specified area by character information to generate a pre-learning model, The transfer learning model generation processing unit includes: generating the transfer learning model by performing transfer learning using second movement trajectory data indicating the position of the target in the specified area by character information and the pre-learning model; The execution processing unit: outputting movement trajectory data indicating a position of a target in the specified area by text information using the transfer learning model; 3. The information processing system according to claim 2.

5. The information processing system includes: a data conversion processing unit that converts the movement line data including the position of the target in the specified area into movement trajectory data represented by text information; 3. The information processing system according to claim 1, further comprising:

6. The data conversion processing unit includes: converting the movement trajectory data output by the execution processing unit into flow line data; The execution processing unit: displaying the flow line data converted by the data conversion processing unit; 6. The information processing system according to claim 5.

7. The movement trajectory data is The specified area is divided into a plurality of areas in a multi-layered manner, and character information is added to each of the divided areas, thereby indicating the position of the target in the specified area with a plurality of pieces of character information.

3. The information processing system according to claim 1 or 2.

8. The movement trajectory data is Character information indicating the position of the target and character information indicating a movement time interval, 8. The information processing system according to claim 7.

9. The movement trajectory data is The character information includes character information indicating the position of the target and character information indicating an action.

8. The information processing system according to claim 7.

10. The movement trajectory data is Character information indicating the position of the target and character information indicating the environment, 8. The information processing system according to claim 7.

11. The machine learning uses a deep learning model or a probabilistic model.

3. The information processing system according to claim 1 or 2.

12. Computer, a pre-learning model generation processing unit that performs machine learning using movement trajectory data indicating the position of a target in a specified region by character information to generate a pre-learning model; an execution processing unit that outputs movement trajectory data using the pre-learning model; An information processing program characterized by causing the program to function as follows.

13. Computer, a transfer learning model generation processing unit that generates a transfer learning model by performing transfer learning using a pre-learning model that has been machine-learned using first movement trajectory data indicating the position of a target in a specified region using character information and second movement trajectory data indicated using character information; an execution processing unit that outputs movement trajectory data using the transfer learning model; An information processing program characterized by causing the program to function as follows.

Images