Person analysis system, person analysis method, and person analysis program

Abstract

To provide a person analysis system capable of detecting the position of each of multiple persons and estimating the behavior using a sensor that emits radio waves to detect a person.SOLUTION: The person analysis system includes: a radar unit at least one of which is installed in a surveillance area and outputs observation data including results of observing the monitoring area using radar; and a person analysis unit that is configured to execute a series of first processing to detect the position of each person present in the monitoring area and a series of second processing to estimate the behavior of each person detected based on the observation data, and to display an area map showing the pictograms of each person present in the monitoring area on a predetermined display unit. The person analysis unit is configured to control the display of pictograms to identify the position and type of action of each person.SELECTED DRAWING: Figure 2 s

Description

[Technical field]

[0001] The present disclosure relates to a person analysis system, a person analysis method, and a person analysis program. [Background technology]

[0002] As a sensor capable of detecting a person in a room while protecting privacy, a sensor that detects a person using microwaves is known. Patent Document 1 discloses that an FMCW (Frequency Modulated Continuous Wave) sensor is used to calculate the sum of the intensities of at least some of the frequency components in the frequency spectrum of a generated beat signal, and if the sum of the intensities is smaller than a predetermined value, it is determined that an object exists in the room, and the position of the person is detected and the state of the person (for example, the state of the person falling down) is estimated based on the spectrum distribution of the generated beat signal. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2016-138796 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, Patent Document 1 does not disclose how to estimate the actions of each of a plurality of people (that is, what each person is doing).

[0005] An object of the present disclosure is to provide a technique for detecting the position and estimating the behavior of each of multiple people by using a sensor that detects people by emitting radio waves. [Means for solving the problem]

[0006] A person analysis system according to one embodiment of the present disclosure includes: a radar device that is installed in at least one monitored area and outputs observation data including results of observing the monitored area using a radar method; and a person analysis device that performs a first process of detecting a position of each person present in the monitored area based on the observation data and a second process of estimating the behavior of each of the detected people, and displays an area map on a specified display device in which pictograms corresponding to each of the people present in the monitored area are placed, and the person analysis device controls the display of the pictograms to identify the position and type of behavior of each person.

[0007] A person analysis method according to one embodiment of the present disclosure includes, in an information processing device, acquiring observation data including results of observing the monitoring area using a radar method from at least one radar device installed in the monitoring area, executing a first process of detecting the position of each person present in the monitoring area based on the observation data, and a second process of estimating the behavior of each of the detected people, displaying an area map on a specified display device in which pictograms corresponding to each of the people present in the monitoring area are placed, and controlling the display of the pictograms to identify the position and type of behavior of each of the people.

[0008] A person analysis program according to one aspect of the present disclosure causes a computer to execute the above-described person analysis method.

[0009] Furthermore, these comprehensive or specific aspects may be realized by a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium, or may be realized by any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium. Effect of the Invention

[0010] According to the present disclosure, by using the results of observing the monitored area using a radar system, the display of pictograms on an area map is controlled to identify the location and type of behavior of each person present in the monitored area, thereby making it possible to see at a glance where each person is located and what behavior they are engaged in. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a person analysis system according to an embodiment of the present invention; [Diagram 2] FIG. 1 is a diagram showing a display example of a display device according to an embodiment of the present invention; [Diagram 3] FIG. 1 is a diagram showing an example of the configuration of a person analysis device according to an embodiment of the present invention; [Figure 4] FIG. 1 is a diagram showing an example of information included in each piece of data according to the present embodiment; [Diagram 5] FIG. 1 is a diagram for explaining an example of analyzing the suitability of a person's behavior according to the present embodiment. [Figure 6] FIG. 1 is a diagram for explaining an example of analyzing a person's physical condition according to the present embodiment; [Figure 7] FIG. 13 is a diagram for explaining an example of displaying a specified action according to the present embodiment. [Figure 8] FIG. 1 is a diagram for explaining a display example of the estimation accuracy of a behavior according to the present embodiment. [Figure 9] FIG. 13 is a diagram showing an example of a screen for setting display conditions for the suitability of an action according to the present embodiment; [Figure 10] FIG. 13 is a diagram showing an example of a screen for setting display conditions for physical condition according to the embodiment; [Figure 11] FIG. 13 is a diagram showing an example of a screen for selecting a use according to the present embodiment; [Figure 12] FIG. 13 is a diagram showing an example of a screen for setting detection sensitivity according to the embodiment; [Figure 13] FIG. 1 is a diagram showing an example of the hardware configuration of an information processing device (computer) that realizes the functional blocks of a person analysis device according to the present disclosure by a computer program; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, the embodiments of the present disclosure will be described in detail with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, detailed description of already well-known matters and duplicated description of substantially the same configuration may be omitted. This is to avoid the following description becoming unnecessarily redundant and to facilitate understanding by those skilled in the art. Note that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0013] (Present embodiment) <Configuration of the Person Analysis System> Fig. 1 is a diagram showing an example of the configuration of a person analysis system 10 according to the present embodiment. Fig. 2 is a diagram showing an example of a display on a display device 50 according to the present embodiment.

[0014] As shown in FIG. 1, the person analysis system 10 includes a radar device 11, a thermosensor 12, an environmental sensor 13, a person analysis device 20, an input device 40, and a display device 50.

[0015] The person analysis device 20 may be able to transmit and receive data to and from at least one of the radar device 11, the thermosensor 12, and the environmental sensor 13 through the communication network 14. Examples of the communication network 14 include a wired LAN (Local Area Network), a wireless LAN, the Internet, or a VPN (Virtual Private Network).

[0016] An input device 40 and a display device 50 are connected to the person analysis device 20. Examples of the input device 40 include a keyboard, a mouse, a touchpad, a microphone, a tablet terminal, etc. Examples of the display device 50 include a liquid crystal display, an organic EL display, a tablet terminal, etc.

[0017] The radar device 11 is an example of a radar type sensor, and at least one is installed in the monitoring area 1. The radar device 11 observes the position of a person 2 or an object (not shown) existing in the monitoring area 1 by irradiating radio waves (radar) and receiving the reflected waves, and generates observation data including the observation results. For example, the radar device 11 transmits radio waves (radar) modulated by FMCW (Frequency Modulated Continuous Wave) to a millimeter wave band (for example, 60 GHz band or 79 GHz band) as a radar type signal (for example, 7 MHz) using a MIMO (Multi Input Multi Output) antenna. Then, the radar device 11 receives the reflected waves of the transmitted radio waves reflected by a person or object using a MIMO antenna. The radar device 11 generates a beat signal, which is an example of observation data, based on a difference signal between the transmitted signal and the received signal. The beat signal may be configured as IQ (In-phase / Quadrature-phase) data. The radar device 11 transmits the IQ data to the person analysis device 20. Note that the radar device 11 may be a LiDAR (Light Detection And Ranging) device.

[0018] At least one thermosensor 12 is installed in the monitoring area 1. The thermosensor 12 measures the body temperature of each person 2 present in the monitoring area 1 by measuring infrared rays emitted from an object, and generates thermodata including the measurement results. The thermosensor 12 transmits the generated thermodata to the person analysis device 20.

[0019] At least one environmental sensor 13 is installed in the monitored area 1. The environmental sensor 13 measures, for example, at least one of the illuminance of the monitored area 1, the illuminance and lighting color of the monitored area 1, the temperature and humidity of the monitored area 1, the noise level of the monitored area 1, and the type of odor of the monitored area 1, and generates environmental data including the measurement results. The environmental sensor 13 transmits the generated environmental data to the person analysis device 20. Note that a different environmental sensor 13 may be used for each measurement target, or one environmental sensor 13 may measure multiple measurement targets.

[0020] The person analysis device 20 executes a first process of detecting the position of each person 2 present in the monitoring area 1 and a second process of estimating the behavior of each detected person 2 based on the observation data received from the radar device 11, and displays information indicating the position and behavior of each person 2 on the display device 50. For example, as shown in FIG. 2, the person analysis device 20 displays an area map 51 corresponding to the monitoring area 1, and displays a pictogram 60 indicating the estimated behavior of the person 2 at the position of the detected person 2 in the area map 51. This allows a user checking the display device 50 to see at a glance where each person 2 present in the monitoring area 1 is located and what behavior he or she is performing. Details of the person analysis device 20 and details of the information displayed on the display device 50 illustrated in FIG. 2 will be described later.

[0021] In the following, an example will be described in which the "monitoring area" is a warehouse, the "persons" are workers working in the warehouse, and the "actions" are typical tasks performed by the workers in the warehouse. In addition, the types of actions will be described by taking as examples the task of transporting luggage (hereinafter referred to as "transport"), the task of collecting luggage (hereinafter referred to as "collection"), the task of packing luggage (hereinafter referred to as "packing"), and the task of inspecting luggage (hereinafter referred to as "inspection").

[0022] In this case, as shown in FIG. 2, the person analysis device 20 displays a pictogram 60 indicating the behavior of the person 2 at a position on the warehouse area map 51 where the person 2 exists. For example, when the person analysis device 20 estimates the behavior of a certain person 2 as "transportation", it displays a pictogram 60A indicating "transportation" at a position on the warehouse area map 51 where the person 2 exists. When the person analysis device 20 estimates the behavior of a certain person 2 as "collection", it displays a pictogram 60B indicating "collection" at a position on the warehouse area map 51 where the person 2 exists. When the person analysis device 20 identifies the behavior of a certain person 2 as "packing", it displays a pictogram 60C indicating "packing" at a position on the warehouse area map 51 where the person 2 exists. When the person analysis device 20 identifies the behavior 2 of a certain person as "inspection", it displays a pictogram 60D indicating "inspection" at a position on the warehouse area map 51 where the person exists.

[0023] This allows a user checking the display device 50 to easily see the location and action of each person 2 working in the warehouse by looking at the pictogram 60 displayed on the display device 50. Note that the pictogram 60 showing the action of a person 2 is not limited to this, and for example, an avatar corresponding to the person may be set and displayed as an animation showing the action of that person.

[0024] The monitored area 1 is not limited to a warehouse, and may be, for example, a factory, a store, a hospital, a convenience store, a department store, an office, a school, a home, or a room, etc. The type of behavior may be any typical behavior performed by a person in the monitored area 1, and the type of behavior may be different when the monitored area 1 is different.

[0025] The person analysis system 10 according to this embodiment will be described in more detail below.

[0026] <Configuration of the Person Analysis Device> Fig. 3 is a diagram showing an example of the configuration of the person analysis device 20 according to the present embodiment. Fig. 4 is a diagram showing an example of information included in each data according to the present embodiment.

[0027] The person analysis device 20 includes, as functions, an IQ data storage unit 21, a thermo data storage unit 22, an environmental data storage unit 23, a point cloud data generation unit 24, a point cloud data storage unit 25, a target data generation unit 26, a target data storage unit 27, a behavioral data generation unit 28, a behavioral data storage unit 29, a vital data generation unit 30, a vital data storage unit 31, and a data analysis unit 32. As shown in Fig. 13, the person analysis device 20 includes at least a processor 1001 and a memory 1002, and the functions of the person analysis device 20 described above may be realized by the processor 1001 reading out a predetermined computer program from the memory 1002 and executing it.

[0028] The IQ data storage unit 21 stores the IQ data periodically transmitted from the radar device 11 .

[0029] The thermodata storage unit 22 stores thermodata periodically transmitted from the thermosensor 12. As shown in Fig. 4, the thermodata may include the time of measurement, a person ID for identifying each person 2, and the measured body temperature of each person 2.

[0030] The environmental data storage unit 23 stores the environmental data periodically transmitted from the environmental sensor 13. As shown in Fig. 4, the environmental data may include at least one of the following: the time of measurement, the position of measurement, information indicating the illuminance and color of the lighting, temperature and humidity, the loudness of noise, and the type of odor.

[0031] <Point cloud data generation section> The point cloud data generating unit 24 generates three-dimensional point cloud data using the IQ data in the IQ data storage unit 21, and stores the data in the point cloud data storage unit 25. As shown in FIG. 4, the point cloud data may include the measurement time and frame number, the position where the point cloud data was measured (i.e., the position of the radar device 11), the position coordinates (x, y, z) of each point, the reflection intensity of each point, and the moving direction and moving speed of each point. Each point included in the point cloud data may be a point whose reflection intensity is equal to or greater than a predetermined threshold (i.e., a point that is estimated not to be noise), or an operating point whose moving speed is equal to or greater than a predetermined threshold. One point indicates the position coordinates (x, y, z) of a reflection point on the surface of a person 2 or an object (not shown). The moving direction and moving speed of a point can be obtained by repeatedly irradiating a modulated radio wave and capturing the reflected wave from multiple directions for a certain period of time, and measuring the Doppler speed. In addition, when LiDAR is used for the radar device 11, the moving speed of a point can be obtained by modulating a laser and measuring the Doppler speed. The information including the position coordinates of each point may be expressed as coordinate information, and the information including the moving direction and moving speed of each point may be expressed as motion information.

[0032] <Target data generation section> The target data generating unit 26 detects the position of each person in the warehouse, the posture and orientation of each person 2, and the density of each person 2, using the point cloud data in the point cloud data storage unit 25. The target data generating unit 26 generates target data based on the detection result and stores it in the target data storage unit 27. As shown in FIG. 4, the target data may include the measurement time, a person ID for identifying the person 2, the position coordinates (x, y, z) of the person 2 in the warehouse, the posture of the person 2, the orientation of the person 2, and the density of the person 2. The density of the person 2 is an index indicating the distance between the person 2 and other people 2. In this embodiment, the types of postures are described using "standing", "sitting", and "lying down" as examples. In addition, in this embodiment, the types of orientation are described using "forward", "sideways", and "diagonally" as examples. In addition, in this embodiment, the types of density are described using "one person", "multiple remote", and "multiple close" as examples. The "single" density indicates that there are no other people 2 within a specified distance from the person 2 in question. The "multiple, distant" density indicates that there are other people 2 within a specified distance from the person 2 in question, but the distance to the other people 2 is equal to or greater than a specified threshold. The "multiple, close" density indicates that there are other people 2 within a specified distance from the person 2 in question, but the distance to the other people 2 is less than a specified threshold.

[0033] For example, the target data generating unit 26 inputs the point cloud data into a DNN (Deep Neural Network) for 3D point clouds, and estimates the position coordinates, posture, and orientation of each person 2 based on the output result from the DNN for the 3D point clouds. The DNN for the 3D point clouds may be trained in advance using point cloud data and a plurality of teacher data in which the position coordinates, posture, and orientation of the person 2 that are correct in the case of the distribution of the point cloud indicated by the point cloud data are associated with each other. Examples of the DNN for the 3D point clouds include PointNet, VoxelNet, PointPillars, and the like. The processing of the target data generating unit 26 may be read as a first processing. The DNN for the 3D point cloud may be read as a first neural network. The target data generating unit 26 includes the estimated position coordinates, posture, and orientation in the target data.

[0034] The target data generating unit 26 may assign a person ID to each person 2 present at each estimated position coordinate, and include the person ID in the target data. The target data generating unit 26 may calculate the distance between each person 2 from the estimated position coordinates of each person 2, and judge the density of each person 2 based on the calculated distance. Then, the target data generating unit 26 may include the result of the density judgment in the target data.

[0035] <Behavioral Data Generation Division> The behavior data generating unit 28 estimates the behavior of the person 2 using the target data in the target data storage unit 27 and the point cloud data in the point cloud data storage unit 25. The behavior data generating unit 28 generates behavior data based on the estimation result and stores the behavior data in the behavior data storage unit 29. For example, as shown in FIG. 4, the behavior data may include a measurement time, a person ID for identifying the person 2, an estimated behavior of the person 2, and a score indicating the accuracy of the estimation (e.g., estimation accuracy or likelihood). In this embodiment, a larger score indicates a higher possibility that the estimated behavior is correct, and a smaller score indicates a lower possibility that the estimated behavior is correct. In this embodiment, as described above, "transport", "collection", "packing", and "inspection" are given as examples of types of behavior.

[0036] For example, the behavior data generation unit 28 identifies the position of person 2 and parts of the person's body (e.g., hands, arms, head, legs, etc.) based on the target data, and generates pseudo image data of parts corresponding to each part of the point cloud data of the person 2. Then, the behavior data generation unit 28 inputs the pseudo image data to a recursive DNN in chronological order, and estimates the type of behavior of person 2 based on the output result from the recursive DNN. For example, the recursive DNN outputs a score for each type of behavior as the output result, and the behavior data generation unit 28 estimates the type of behavior with the highest output score to be the behavior of the person 2.

[0037] The recursive DNN may be trained in advance using a plurality of teacher data that associates the time change of pseudo image data with the behavior of a person that is the correct answer when the pseudo image data changes over time. Examples of the recursive DNN include LSTM (Long Short Term Memory) and GRU (Gated Recurrent Unit). The process of the behavior data generating unit 28 may be read as the second process. The recursive DNN may be read as the second neural network.

[0038] <Vital Data Generation Division> The vital data generation unit 30 estimates the vitals of the person 2 using the target data in the target data storage unit 27 and the IQ data in the IQ data storage unit 21. The vital data generation unit 30 generates vital data based on the estimation result and stores it in the vital data storage unit 31. As shown in Fig. 4, the vital data may include the measurement time, a person ID for identifying the person 2, an estimated respiratory rate, an estimated heart rate, and a score indicating the accuracy of the estimation (e.g., estimation accuracy or likelihood).

[0039] For example, the vital data generating unit 30 identifies the position of each person 2 based on the target data, and identifies the chest of the person 2 present at the identified position. Then, the vital data generating unit 30 calculates slight periodic motion of the body surface of the person 2 based on a time change in phase information of a portion of the IQ data corresponding to the chest, and estimates the respiratory rate and heart rate based on the periodic motion. The vital data generating unit 30 may estimate the respiratory rate and heart rate by performing spectrum analysis and / or deep learning analysis.

[0040] For example, the vital data generating unit 30 may calculate a score of the estimated respiratory rate and / or heart rate based on a signal-noise ratio (SN ratio) related to the IQ data. For example, the vital data generating unit 30 increases the score as the SN ratio increases, and decreases the score as the SN ratio decreases. This is because when the SN ratio is small, noise components are large and the estimation accuracy may decrease. The vital data generating unit 30 includes the calculated score in the vital data.

[0041] The vital data generating unit 30 may estimate blood pressure and the like using a similar method. In this case, the vital data shown in FIG.

[0042] <Data Analysis Department> The data analysis unit 32 performs various analyses on each person 2 using at least one of the target data in the target data storage unit 27, the behavioral data in the behavioral data storage unit 29, and the vital data in the vital data storage unit 31. Furthermore, when performing the analysis, the data analysis unit 32 may further use the thermo data in the thermo data storage unit 22 and / or the environmental data in the environmental data storage unit 23. Next, an example of analysis by the data analysis unit 32 will be described.

[0043] <Analysis of the appropriateness of the action> FIG. 5 is a diagram for explaining an example of analyzing the appropriateness of the behavior of person 2 according to the present embodiment.

[0044] The data analysis unit 32 identifies the behavior of the person 2 from the behavior data in the behavior data storage unit 29. The data analysis unit 32 identifies the posture and orientation of the person 2 from the target data in the target data storage unit 27. The data analysis unit 32 judges whether the behavior of the person 2 is appropriate based on the identified posture and orientation for the identified behavior.

[0045] For example, it is assumed that the ideal posture and orientation when the person 2 performs the action "packing" are "standing" and "forward," respectively. In this case, the data analysis unit 32 performs, for example, the following processes (A1) to (A3).

[0046] (A1) When the data analysis unit 32 identifies the behavior of the person 2 as "packing", the posture of the person as "standing", and the orientation of the person 2 as "forward", these match the ideal posture and orientation for the behavior described above, and therefore the behavior of the person 2 is determined to be "appropriate". Then, as shown in Fig. 5(a), the data analysis unit 32 displays a pictogram 60C indicating the behavior of the person 2 as "packing" together with a "circle" mark 61A indicating that the behavior is "appropriate".

[0047] (A2) When the data analysis unit 32 identifies the behavior of the person 2 as "packing", the posture of the person 2 as "standing", and the orientation of the person 2 as "diagonal", the behavior of the person 2 matches the ideal posture described above for the behavior but does not match the ideal orientation, and thus determines the behavior of the person 2 as "slightly inappropriate". Then, as shown in FIG. 5(b), the data analysis unit 32 displays a "△" mark 61B indicating that the behavior is "slightly inappropriate" together with a pictogram 60C indicating the behavior of the person 2 "packing". At this time, the data analysis unit 32 may display the color of the pictogram 60C indicating the behavior of the person 2 "packing" in a color that is previously associated with the behavior being "slightly inappropriate".

[0048] (A3) When the data analysis unit 32 identifies the behavior of person 2 as "packing", the posture of person 2 as "sitting", and the orientation of person 2 as "diagonal", these do not match any of the ideal postures and orientations described above for the behavior, and so determines the behavior of person 2 as "inappropriate". Then, as shown in FIG. 5(c), the data analysis unit 32 displays a "x" mark 61C indicating that the behavior is "inappropriate" together with a pictogram 60C indicating the behavior of person 2 "packing". At this time, the data analysis unit 32 may display the color of the pictogram 60C indicating the behavior of person 2 "packing" in a color that is previously associated with the behavior being "inappropriate".

[0049] This allows the user to easily see the pictograms 60 and marks 61 displayed on the display device 50 to see where each person 2 working in the warehouse is located and what action each person 2 is taking, and whether each person 2 is taking appropriate action.

[0050] The term "pictogram" is an example, and any type of information that can distinguish the type of behavior may be used, such as an illustration, photograph, symbol, or sign. The term "mark" is an example, and any type of information that can distinguish the appropriateness of the behavior may be used, such as a letter, number, symbol, pattern, or pictogram.

[0051] <Physical condition analysis> FIG. 6 is a diagram for explaining an example of analyzing the physical condition of the person 2 according to the present embodiment.

[0052] The data analysis unit 32 identifies the behavior of the person 2 from the behavior data in the behavior data storage unit 29. The data analysis unit 32 identifies the posture of the person 2 from the target data in the target data storage unit 27. The data analysis unit 32 identifies the time change in the respiratory rate of the person 2 and the time change in the heart rate of the person 2 from the vital data in the vital data storage unit 31. The data analysis unit 32 judges the physical condition of the person at the time of the behavior based on the identified posture and the identified time change in the respiratory rate and the heart rate for the identified behavior.

[0053] For example, the normal posture of the person 2 when performing the action "inspection" is "standing," and the normal changes in the respiration rate and the heart rate over time are relatively small (i.e., stable). Here, the relatively small change in the respiration rate over time means that the respiration rate is within a predetermined normal range (e.g., between an upper threshold and a lower threshold) at a predetermined time, and the relatively large change in the respiration rate over time may mean that the respiration rate is not within the normal range (e.g., greater than the upper threshold or less than the lower threshold). Similarly, the relatively small change in the heart rate over time means that the heart rate is within a predetermined normal range (e.g., between an upper threshold and a lower threshold) at a predetermined time, and the relatively large change in the heart rate over time may mean that the heart rate is not within the normal range (e.g., greater than the upper threshold or less than the lower threshold). In this case, the data analysis unit 32 performs, for example, the following processes (B1) to (B3).

[0054] (B1) If the data analysis unit 32 identifies the behavior of person 2 as "inspection", the posture of person 2 as "standing", and the time change in the respiratory rate and heart rate of person 2 as being relatively small (i.e. stable), this matches the above-mentioned normal posture for that behavior and the above-mentioned normal respiratory rate and heart rate for that behavior, and therefore determines the physical condition of person 2 to be "normal".The data analysis unit 32 then displays a pictogram 60D indicating the behavior of person 2 of "inspection", together with a "◎" mark 62A indicating that the physical condition is "normal".

[0055] (B2) When the data analysis unit 32 identifies the behavior of the person 2 as "inspection", identifies the posture of the person 2 as "standing", identifies the time change of the respiratory rate of the person 2 as relatively small (i.e. stable), and identifies the time change of the heart rate of the person 2 as relatively large (for example, the heart rate repeatedly rises and falls and is unstable), the behavior matches the above-mentioned normal posture for the behavior, and matches the above-mentioned normal respiratory rate for the behavior, but does not match the above-mentioned normal heart rate for the behavior, and therefore determines the physical condition of the person 2 as "needs a break". Then, the data analysis unit 32 displays the pictogram 60D indicating the behavior of the person 2 as "inspection", together with the "☆" mark 62B indicating that the physical condition is "needs a break". At this time, the data analysis unit 32 may display the color of the person's pictogram 60D in a color that is previously associated with "needs a break".

[0056] (B3) When the data analysis unit 32 identifies the behavior of the person 2 as "inspection", identifies the posture of the person 2 as "sitting position", identifies the time change of the respiratory rate of the person 2 as relatively large (for example, the respiratory rate is rising rapidly and is unstable), and identifies the time change of the heart rate of the person 2 as relatively large (for example, the heart rate is rising rapidly and is unstable), the behavior does not match the above-mentioned normal posture for the behavior, and does not match either the above-mentioned normal respiratory rate or heart rate for the behavior, and thus determines the physical condition of the person 2 as "requires medical assistance". Then, the data analysis unit 32 displays the pictogram 60D indicating the behavior of the person 2 as "inspection", together with the "!" mark 62C indicating that the physical condition is "requires medical assistance". At this time, the data analysis unit 32 may display the color of the pictogram 60D of the person 2 in a color that is previously associated with "requires medical assistance".

[0057] This allows the user to easily see the pictograms 60 and marks 62 displayed on the display device 50 to ascertain where each person working in the warehouse is located, what actions they are taking, and what physical condition each person is in.

[0058] It should be noted that the term "mark" is merely an example, and any mark that can distinguish the physical condition may be used, such as letters, numbers, symbols, patterns, or pictograms.

[0059] <Display of configured actions> FIG. 7 is a diagram for explaining an example of displaying a set action according to the present embodiment.

[0060] The user sets display conditions related to behavior in advance in the data analysis unit 32 through the input device 40. The data analysis unit 32 identifies behavior of the person 2 from the behavior data in the behavior data storage unit 29. If the identified behavior satisfies the display conditions set above, the data analysis unit 32 displays a pictogram 60 indicating the behavior, and if the identified behavior does not satisfy the display conditions set above, the data analysis unit 32 does not display a pictogram 60 indicating the behavior. A specific example will be described below.

[0061] For example, the user sets the display conditions to display the action "inspection" and not display other actions through the input device 40. Then, the data analysis unit 32 estimates the action of the first person 2 as "collection" as shown in Fig. 7(a), the action of the second person 2 as "packing" as shown in Fig. 7(b), and the action of the third person 2 as "inspection" as shown in Fig. 7(c).

[0062] In this case, as shown in Figure 7(c), the data analysis unit 32 displays a pictogram 60D indicating the action of "inspection" on the area map 51 for the third person 2 whose action was estimated to be "inspection," and does not display a pictogram 60 on the area map 51 for the first person 2 whose action was estimated to be "collecting" and the second person whose action was estimated to be "packing," as shown in Figures 7(a) and (b).

[0063] This allows the user to easily check the location of each person 2 who is performing the behavior set by the user as a display condition by looking at the pictogram 60 displayed on the display device 50. For example, when there are many people 2 in the monitored area 1 and the display of the area map 51 becomes cluttered, this setting can eliminate the clutter of the display.

[0064] <Display of estimated behavior accuracy> FIG. 8 is a diagram for explaining a display example of the estimation accuracy of a behavior according to the present embodiment.

[0065] The data analysis unit 32 identifies the behavior of the person 2 from the behavior data in the behavior data storage unit 29. The data analysis unit 32 displays on the display device 50 a pictogram 60 indicating the identified behavior.

[0066] At this time, when the behavioral data includes a plurality of behaviors each having a score equal to or higher than a predetermined threshold, the data analysis unit 32 may display the pictograms 60 each representing the plurality of behaviors together. In addition, the data analysis unit 32 may display the pictogram 60 representing the behavior with the highest score among the plurality of pictograms 60 in a color or brightness different from that of the pictograms 60 representing the other behaviors.

[0067] Furthermore, when the behavior data does not include a behavior with a score equal to or higher than a predetermined threshold, the data analysis unit 32 may display pictograms 60 indicating one or more behaviors with the highest scores in a color or brightness different from that of the pictograms 60 displayed when the score is equal to or higher than a predetermined threshold. A specific example will be described below.

[0068] For example, as shown in FIG. 8(a), in the behavior data of a certain person in the behavior data storage unit 29, the score of the behavior "transport" is 0.8, the score of the behavior "collection" is 0.7, and the density of the person 2 is "alone". In this case, the scores of the behavior "transport" and the behavior "collection" are both equal to or higher than a predetermined threshold (e.g., 0.7), and it is difficult to determine which is the correct answer. In this case, as shown in FIG. 8(a), the data analysis unit 32 may display both the pictogram 60A indicating the behavior "transport" and the pictogram 60B indicating the behavior "collection" on the area map 51. Then, the data analysis unit 32 may display the pictogram 60A indicating the behavior "transport" with a relatively higher score with a lower brightness (darker) and the pictogram 60B indicating the behavior "collection" with a relatively lower score with a higher brightness (lighter). Furthermore, the data analysis unit 32 may display a pictogram 63A indicating that the density of the person 2 is "alone" together with the pictogram 60 indicating the behavior.

[0069] This allows the user to easily check the estimation accuracy of the behavior of each person 2 by looking at the pictogram 60 displayed on the display device 50.

[0070] For example, as shown in FIG. 8(b), in the behavior data of a certain person 2 in the behavior data storage unit 29, the score of the behavior "transport" is 0.8, the score of the behavior "collection" is 0.7, and the density of the person 2 is "multiple remote". In this case, the scores of the behavior "transport" and the behavior "collection" are both equal to or higher than a predetermined threshold (e.g., 0.7), and it is difficult to determine which is the correct answer. In this case, as shown in FIG. 8(b), the data analysis unit 32 may display a pictogram 60A indicating the behavior "transport" and a pictogram 60B indicating the behavior "collection" together on the area map 51. Then, the data analysis unit 32 may display the pictogram 60A indicating the behavior "transport" with a relatively high score with a lower brightness (darker) and the pictogram 60B indicating the behavior "collection" with a relatively low score with a higher brightness (lighter). Furthermore, the data analysis unit 32 may display a pictogram 63B indicating that the density of the person 2 is "multiple, remote" together with the pictogram 60 indicating the behavior.

[0071] This allows the user to easily check the accuracy of estimation of the behavior of the person 2 by looking at the pictogram 60 displayed on the display device 50. Furthermore, when the user looks at the pictogram 63B indicating that the density of the person 2 is "multiple remote", the user can assume that the accuracy of estimation of the behavior may be lower than when the pictogram 63A indicating that the density of the person 2 is "one" is displayed.

[0072] For example, as shown in FIG. 8(c), in the behavior data in the behavior data storage unit 29, the score of the behavior "transport" is 0.6, the score of the behavior "collection" is 0.5, and the density of the person 2 is "multiple proximity". In this case, the scores of the behavior "transport" and the behavior "collection" are both less than a predetermined threshold (e.g., 0.7), and it is difficult to determine which is the correct answer, and it is possible that neither is correct. In this case, as shown in FIG. 8(c), the data analysis unit 32 may display the two highest-ranked pictograms 60A indicating the behavior "transport" and 60B indicating the behavior "collection" together on the area map 51. Then, since both scores are relatively low, the data analysis unit 32 may display the brightness of both the pictogram 60A indicating the behavior "transport" and the pictogram 60B indicating the behavior "collection" low (light). Furthermore, the data analysis unit 32 may display a pictogram 63C indicating that the density of the persons 2 is "multiple persons in close proximity" together with the pictogram 60 indicating the behavior.

[0073] This allows the user to easily check the estimation accuracy of the person's behavior by looking at the pictogram 60 displayed on the display device 50. Furthermore, when the user looks at the pictogram 63C indicating that the density of person 2 is "multiple close", the user can assume that the estimation accuracy of the behavior may be lower than when the pictograms 63A and 63B indicating that the density of people is "one" or "multiple distant" are displayed.

[0074] <Display setting screen for whether or not an action is appropriate> FIG. 9 is a diagram showing an example of a screen for setting display conditions for the appropriateness of an action according to the present embodiment.

[0075] 9, the data analysis unit 32 may display a screen for setting display conditions for the suitability of an action (hereinafter referred to as a suitability of an action display setting screen 100) on the display device 50. The suitability of an action display setting screen 100 includes an action selection area 101, a posture selection area 102, an orientation selection area 103, a density selection area 104, a time selection area 105, and a pictogram and mark set selection area 106.

[0076] For example, assume that the user selects "packing" in the action selection area 101A, "standing" in the posture selection area 102A, "in front" in the orientation selection area 103A, "alone" in the density selection area 104A, "5 seconds" in the time selection area 105A, and selects a set of a pictogram 60C indicating "packing" and a "circle" mark 61A indicating that the action is "appropriate" in the pictogram and mark set selection area 106A, as shown in Fig. 9(a). In this case, when the state where the action of a certain person is specified as "packing", the posture is specified as "standing", the orientation is specified as "in front", and the density of the person 2 is specified as "alone" continues for "5 seconds" or more, the data analysis unit 32 displays the pictogram 60C indicating the action "packing" and the "circle" mark 61A indicating that the action is "appropriate" together according to the display conditions of Fig. 9(a).

[0077] For example, assume that the user selects "packing" in the action selection area 101B, "standing" in the posture selection area 102B, "diagonal" in the orientation selection area 103B, "alone" in the density selection area 104B, and "5 seconds" in the time selection area 105B through the input device 40, and selects a set of a pictogram 60C indicating "packing" and a "△" mark 61B indicating that the action is "slightly inappropriate" in a pictogram and mark set selection area 106B, as shown in Fig. 9(b). In this case, when the state where the action of a certain person 2 is specified as "packing", the posture is specified as "standing", the orientation is specified as "diagonal", and the density of the person 2 is specified as "alone" continues for "5 seconds" or more, the data analysis unit 32 displays the pictogram 60C indicating the action "packing" and the "△" mark 61B indicating that the action is "slightly inappropriate" together according to the display conditions of Fig. 9(b).

[0078] In this way, the user can arbitrarily set the display conditions for the suitability of an action.

[0079] <Health display setting screen> FIG. 10 is a diagram showing an example of a screen for setting display conditions of physical condition according to the present embodiment.

[0080] The data analysis section 32 may display on the display device 50 a screen for setting the display conditions of the physical condition (hereinafter referred to as a physical condition display setting screen 120), as shown in FIG.

[0081] Displayed on the physical condition display setting screen 120 are a vital sign selection area 121, a vital sign state selection area 122, a posture selection area 123, a time selection area 124, and a pictogram and mark set selection area 125. As shown in Fig. 10, two or more sets of vital sign selection area 121 and vital sign state selection area 122 may be provided.

[0082] For example, suppose that a user, through input device 40, selects "respiratory rate" in selection area 121A of the first set of vital signs, selects "stable" in selection area 122A of the status of the first set of vital signs, selects "heart rate" in selection area 121B of the second set of vital signs, selects "stable" in selection area 122B of the status of the second set of vital signs, selects "standing" in posture selection area 123A, selects "10 seconds" in time selection area 124A, and selects a set of pictogram 60D indicating "inspection" and "◎" mark 62A indicating "normal" in selection area 125A of the set of pictograms and marks. In this case, as shown in FIG. 6(a), if the data analysis unit 32 identifies a person's behavior as "inspection", the posture as "standing", the change in respiratory rate over time as "stable", and the change in heart rate over time as "stable" for a period of "10 seconds" or more, it displays a pictogram 60D indicating the behavior "inspection" together with a "◎" mark 62A indicating that the physical condition is "normal", in accordance with the display conditions of FIG. 10(a).

[0083] For example, suppose that the user, through input device 40, selects "respiratory rate" in selection area 121C of the first set of vital signs, selects "stable" in selection area 122C of the first set of vital signs status, selects "heart rate" in selection area 121D of the second set of vital signs, selects "unstable" in selection area 122D of the second set of vital signs status, selects "standing" in posture selection area 123B, selects "10 seconds" in time selection area 124B, and selects a set of pictogram 60D indicating "inspection" and "☆" mark 62B indicating "break needed" in selection area 125B of the pictogram and mark set. In this case, as shown in Figure 6(b), if the data analysis unit 32 identifies a person's behavior as "inspection", the posture as "standing", the change in respiratory rate over time as "stable", and the change in heart rate over time as "unstable" for a period of "10 seconds" or more, it will display a pictogram 60D indicating the behavior "inspection" together with a "☆" mark 62B indicating that the physical condition is "needs a break", in accordance with the display conditions of Figure 10(b).

[0084] Instead of selecting the state "stable", a range of vital values ​​corresponding to "stable" (e.g., upper and lower thresholds of the range) may be input. Also, instead of selecting the state "unstable", an upper and lower thresholds for determining an unstable state may be input. In this case, if the vital value is greater than the upper threshold or less than the lower threshold, it may be determined to be unstable.

[0085] In this way, the user can arbitrarily set the conditions for displaying the physical condition.

[0086] <Application selection screen> FIG. 11 is a diagram showing an example of a screen for selecting a use according to the present embodiment.

[0087] The data analysis section 32 may display on the display device 50 a screen for selecting the purpose of the person analysis system 10 (hereinafter, referred to as a purpose selection screen 140), as shown in FIG.

[0088] A use selection list 141 is displayed on the use selection screen 140. Each use is associated in advance with the above-mentioned display conditions suitable for that use.

[0089] For example, when the user selects one use from the use selection list 141 through the input device 40 as shown in FIG. 11, the data analysis unit 32 reads out and sets the display conditions that are pre-associated with the selected use.

[0090] This allows the user to set the display conditions more easily.

[0091] <Detection sensitivity setting screen> FIG. 12 is a diagram showing an example of a screen for setting the detection sensitivity according to the present embodiment.

[0092] The data analysis unit 32 displays on the display device 50 a screen for selecting the detection sensitivity of the behavior (hereinafter, referred to as a detection sensitivity selection screen 160).

[0093] A selection list 161 of detection sensitivity levels is displayed on the detection sensitivity selection screen 160. For example, as shown in Fig. 12, the selection list 161 allows the user to select from three detection sensitivity levels: "high", "normal", and "low".

[0094] The data analysis unit 32 determines the display sensitivity of the pictogram 60 indicating the behavior set as the display target shown in FIG. 7 according to the selected level of detection sensitivity.

[0095] For example, when a user selects a detection sensitivity of "high" from the detection sensitivity selection list 161 through the input device 40, a first threshold value that is pre-associated with the detection sensitivity of "high" is set as the threshold value for the score of the behavior set as the display target.

[0096] For example, when a user selects the detection sensitivity "normal" from the detection sensitivity selection list 161 through the input device 40, a second threshold value that is pre-associated with the detection sensitivity "normal" is set as the threshold value for the score of the behavior set as the display target.

[0097] For example, when a user selects the detection sensitivity "low" from the detection sensitivity selection list 161 through the input device 40, a third threshold value that is pre-associated with the detection sensitivity "low" is set as the threshold value for the score of the behavior set as the display target.

[0098] Here, the first threshold is smaller than the second threshold, and the second threshold is smaller than the third threshold.

[0099] As a result, for example, when the detection sensitivity is selected as "high", even if the score of the behavior is relatively low (for example, a score smaller than the second threshold value but larger than the first threshold value), the data analysis unit 32 determines to display the pictogram 60 indicating the behavior in the display of the set behavior shown in Fig. 7. In other words, the higher the detection sensitivity, the more likely it is that the pictogram 60 indicating the set behavior will be displayed, and the lower the detection sensitivity, the less likely it is that the pictogram 60 indicating the set behavior will be displayed.

[0100] In this way, the user can arbitrarily set the detection sensitivity regarding the behavior of the person 2. In other words, the user can arbitrarily set the display sensitivity of the pictogram 60 indicating the behavior.

[0101] <Estimation of a person's mental state> The data analysis unit 32 may estimate the mental state of the person 2 based on the behavioral data in the behavioral data storage unit 29 and the vital data in the vital data storage unit 31. For example, when the data analysis unit 32 identifies the posture as "standing" from the behavioral data and identifies a sudden rise in the heart rate from the vital data, the data analysis unit 32 may estimate the mental state of the person 2 as "high stress". Furthermore, when estimating the mental state of the person 2, the data analysis unit 32 may use at least one of the thermo data and the environmental data to improve the estimation accuracy. For example, when the data analysis unit 32 identifies a sudden rise in the body temperature of the person 2 from the thermo data and identifies the temperature in the warehouse as an uncomfortable temperature (for example, a temperature above a predetermined threshold) from the environmental data, the data analysis unit 32 may calculate the estimation accuracy of the mental state of the person 2 as "high stress" to be higher than the estimation accuracy when the mental state is "high stress" without using the thermo data and the environmental data.

[0102] In this way, by estimating the mental state of the person 2 using not only information obtained from the radar device 11 but also information obtained from sensors other than the radar device 11, such as the thermosensor 12 or the environmental sensor 13, the accuracy of the estimation can be improved. Note that examples of the mental state of the person 2 include the level of "drowsiness", the level of "tension", the level of "relaxation", etc., in addition to the level of "stress" described above.

[0103] (Hardware configuration) The above-mentioned functional blocks of the person analysis device 20 can be realized by a computer program.

[0104] FIG. 13 is a diagram showing an example of the hardware configuration of an information processing device (computer) that realizes the functional blocks of the person analysis device 20 according to the present disclosure by a computer program.

[0105] The information processing device 1000 includes a processor 1001, a memory 1002, a storage 1003, an input I / F (Interface) 1004, an output I / F 1005, a communication I / F 1006, a GPU (Graphics Processing Unit) 1007, a reading I / F 1008, and a bus 1009.

[0106] The processor 1001, memory 1002, storage 1003, input I / F 1004, output I / F 1005, communication I / F 1006, GPU (Graphics Processing Unit) 1007, and read I / F 1008 are connected to a bus 1009, and can send and receive data in both directions via the bus 1009.

[0107] The processor 1001 is a device that executes a computer program stored in the memory 1002 and realizes the above-mentioned functional blocks. Examples of the processor 1001 include a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a controller, an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field-Programmable Gate Array).

[0108] The memory 1002 is a device that stores computer programs and data that are handled by the information processing device 1000. The memory 1002 may include a read-only memory (ROM) and a random access memory (RAM).

[0109] The storage 1003 is configured as a non-volatile storage medium, and is a device that stores computer programs and data handled by the information processing device 1000. Examples of the storage 1003 include a hard disk drive (HDD) and a solid state drive (SSD).

[0110] The input I / F 1004 is connected to the input device 40 that accepts input from a user, and transmits data received from the input device 40 to the processor 1001 .

[0111] The output I / F 1005 is connected to the display device 50 and transmits data received from the processor 1001 to the display device 50 .

[0112] The communication I / F 1006 is connected to the communication network 14, and transmits and receives data to and from other devices via the communication network 14. The communication I / F 1006 may support either wired communication or wireless communication. An example of wired communication is Ethernet (registered trademark). Examples of wireless communication are Wi-Fi (registered trademark), Bluetooth (registered trademark), LTE (Long Term Evolution), 4G, and 5G.

[0113] The GPU 1007 is a device that performs high-speed image rendering processing. The GPU 1007 may be used for AI (Artificial Intelligence) processing (e.g., deep learning processing).

[0114] An external storage medium is connected to the read I / F 1008, and data is read from the external storage medium. Examples of the external storage medium include a digital versatile disk read only memory (DVD-ROM) and a universal serial bus (USB) memory.

[0115] The functional blocks of the person analysis device 20 may be realized as an LSI, which is an integrated circuit. These functional blocks may be individually integrated into one chip, or some or all of them may be integrated into one chip. Here, it is referred to as an LSI, but depending on the degree of integration, it may be called an IC, a system LSI, a super LSI, or an ultra LSI. Furthermore, if an integrated circuit technology that can replace LSI appears due to the progress of semiconductor technology or a different derived technology, it is natural that the functional blocks may be integrated using that technology.

[0116] Summary of the Disclosure The above description of the embodiments discloses the following techniques.

[0117] <Technology 1> The person analysis system 10 comprises: a radar device 11, which is installed at least in a monitored area 1 and outputs observation data including the results of observing the monitored area 1 by a radar method; and a person analysis device 20 which executes a first process (e.g., a process by a target data generation unit 26) to detect the position of each person 2 present in the monitored area 1 based on the observation data, and a second process (e.g., a process by a behavior data generation unit 28) to estimate the behavior of each of the detected people 2, and displays information indicating the position and behavior of each person 2 on a specified display device 50. This makes it possible to display the location of each person 2 in the monitored area 1 and the behavior of each person 2 while protecting privacy.

[0118] <Technology 2> In the person analysis system 10 described in Technique 1, the person analysis device 20 generates three-dimensional point cloud data based on the observation data, and in the first processing, detects the position of each person 2 based on the distribution of the point cloud data, and in the second processing, estimates the behavior of each person 2 based on changes in the point cloud data over time. This makes it possible to detect the position of each person 2 through the first processing, and to estimate the behavior of each person 2 through the second processing.

[0119] <Technology 3> In the person analysis system 10 described in Technology 1 or Technology 2, the first process detects the position of each person 2 using a first neural network that has been trained in advance to output the position where person 2 is located when the point cloud data is input, and the second process estimates the behavior of each person 2 using a second neural network that has been trained in advance to output the behavior of person 2 when the change in the point cloud data over time is input. This makes it possible to detect the position of each person 2 using the first neural network, and to estimate the behavior of each person 2 using the second neural network.

[0120] <Technology 4> In the person analysis system 10 described in any one of Techniques 1 to 3, the point cloud data includes at least coordinate information indicating three-dimensional coordinates of each point and motion information indicating a direction and a speed of movement of each point, and the second process estimates the behavior of the person 2 by inputting at least the motion information into the second neural network. In this way, by estimating the behavior of the person 2 using the movement information, the accuracy of estimating the behavior is improved.

[0121] <Technology 5> In the person analysis system 10 described in any one of Techniques 2 to 4, the person analysis device 20 displays an area map 51 corresponding to the surveillance area 1, and also displays a pictogram 60 indicating the estimated behavior of the detected person 2 at the position of the detected person 2 on the area map 51. This allows the user to easily see the pictograms 60 indicating the displayed actions and to confirm where each person 2 is located and what action he or she is taking.

[0122] <Technology 6> In the person analysis system 10 described in any one of Techniques 2 to 5, in the first process, the person analysis device 20 further detects the posture and orientation of the person 2 based on the distribution of the point cloud data, judges the appropriateness of the behavior of the person 2 based on the estimated behavior of the person 2 and the detected posture and orientation of the person 2, and displays a pictogram 60 indicating the behavior of the person 2 together with information indicating the judgment result of the appropriateness of the behavior of the person 2. This allows the user to easily check what action each person 2 is performing, where he or she is located, and whether or not each person 2 is performing appropriate behavior, by looking at the pictogram 60 indicating the displayed behavior and the information indicating the judgment result of whether or not the behavior is appropriate.

[0123] <Technology 7> In the person analysis system 10 described in any one of Techniques 2 to 6, the person analysis device 20 estimates vital signs of the person 2 based on the point cloud data and the observation data, judges a physical condition of the person based on the estimated behavior of the person 2, the detected posture of the person 2, and the estimated vital signs of the person 2, and displays a pictogram 60 indicating the behavior of the person 2 together with information indicating the judgment result of the physical condition of the person. This allows the user to easily check what action each person 2 is taking, where they are located, and how their physical condition is by looking at the displayed pictogram 60 indicating the action and the information indicating the result of the physical condition assessment.

[0124] <Technology 8> In the person analysis system 10 described in any one of Techniques 2 to 7, the person analysis device 20 displays a pictogram 60 indicating the behavior of the person 2 when the estimated behavior of the person 2 meets preset behavior display conditions, and does not display the pictogram 60 indicating the behavior of the person 2 when the estimated behavior of the person 2 does not meet the display conditions. As a result, pictograms 60 that do not meet the display conditions previously set by the user are not displayed, and pictograms 60 that meet the display conditions are displayed, so that the user can quickly recognize people who are performing actions that meet the display conditions.

[0125] <Technology 9> In the person analysis system 10 described in Technique 3, the second neural network is configured to output a score indicating the likelihood of estimating the behavior for each type of behavior, and when there are multiple behaviors for which the score is equal to or greater than a predetermined threshold, the person analysis device 20 displays pictograms 60 indicating each of the multiple behaviors together. This allows the user to see the multiple pictograms 60 displayed and recognize that there is a high possibility that the person 2 is taking any of the actions indicated by the multiple pictograms 60.

[0126] <Technology 10> In the person analysis system 10 described in Technique 9, the person analysis device 20 displays the pictogram 60 indicating the behavior with the highest score among the plurality of behaviors in a color or brightness different from that of the pictograms 60 indicating the other behaviors. This allows the user to see at a glance which of the multiple displayed pictograms 60 has the highest estimation accuracy.

[0127] <Technology 11> In the person analysis system 10 described in Technology 3, the second neural network is configured to output a score indicating the likelihood of estimating the behavior for each type of behavior, and when there is no behavior for which the score is equal to or greater than a predetermined threshold, the person analysis device 20 displays pictograms 60 indicating one or more behaviors ranked from the top of the score in a color or brightness different from that of the pictograms 60 displayed when the score is equal to or greater than the predetermined threshold. This allows the user to see the multiple pictograms 60 displayed and recognize at a glance that the estimation accuracy of the action indicated by the pictogram 60 may be low.

[0128] <Technology 12> In the person analysis system 10 described in any one of Techniques 1 to 11, the person analysis device 20 identifies the closeness of each person 2 to other people 2 based on the position of each detected person 2, and displays information indicating the closeness of the person 2 together with a pictogram 60 indicating the behavior of the person 2. This allows the user to see the displayed pictogram 60 and the information indicating the density of people, and to grasp the level of accuracy of the estimation of the behavior indicated by the pictogram.

[0129] <Technology 13> The person analysis method acquires observation data including the results of observing the monitored area 1 by a radar method from at least one radar device 11 installed in the monitored area 1, and executes a first process (e.g., processing by the target data generation unit 26) to detect the position of each person 2 present in the monitored area 1 based on the observation data, and a second process (e.g., processing by the behavior data generation unit 28) to estimate the behavior of each of the detected people 2, and displays information indicating the position and behavior of each person 2 on a specified display device 50. This makes it possible to display the location of each person 2 in the monitored area 1 and the behavior of each person 2 while protecting privacy.

[0130] <Technology 14> The person analysis program causes a computer to execute the person analysis method described in technique 13. This makes it possible to display the location of each person 2 in the monitored area 1 and the behavior of each person 2 while protecting privacy.

[0131] Although the embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications, corrections, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also belong to the technical scope of the present disclosure. In addition, the components in the above-mentioned embodiments may be arbitrarily combined within the scope of the invention. [Industrial Applicability]

[0132] The technology disclosed herein is useful for analyzing a person's location, behavior, etc. while protecting privacy. [Explanation of symbols]

[0133] 1 Surveillance Area 2 people 10. Person Analysis System 11 Radar equipment 12 Thermosensor 13 Environmental Sensors 14. Communications Networks 20 Person analysis device 21 IQ data storage section 22 Thermo data storage section 23 Environmental Data Storage Unit 24 Point cloud data generator 25 Point cloud data storage section 26 Target data generation unit 27 Target data storage section 28 Behavioral Data Generation Unit 29 Behavioral Data Storage 30 Vital Data Generation Unit 31 Vital Data Storage Unit 32 Data Analysis Department 40 Input Devices 50 Display device 51 Area Map 60, 60A, 60B, 60C, 60D pictograms 61, 61A, 61B, 61C Mark 62, 62A, 62B, 62C Mark 63A, 63B, 63C pictograms 100 Action suitability display setting screen 101 Choice of Actions 102 Posture Selection Area 103 Orientation Selection Area 104 Selection area of ​​sparseness and density 105 Time Selection Area 106 Selection area for sets of pictograms and marks 120 Physical condition display setting screen 121, 121A, 121B Vital Selection Areas 122, 122A, 122B Vital Status Selection Area 123 Posture Selection Area 124 hour selection area 125 Selection area for sets of pictograms and marks 140 Use selection screen 141 Use Selection List 160 Detection sensitivity selection screen 161 Selection list for high detection sensitivity

Claims

1. At least one radar device is installed in a monitoring area and outputs observation data including results of observing the monitoring area by a radar system; a person analysis device that executes a first process of detecting a position of each person present in the monitoring area based on the observation data and a second process of estimating a behavior of each of the detected people, and displays an area map on a predetermined display device, in which pictograms corresponding to each of the people present in the monitoring area are arranged; the person analysis device controls display of the pictograms to identify the position and the type of behavior of each person; People analysis system.

2. The person analysis device includes: generating three-dimensional point cloud data based on the observation data; In the first process, a position of each person is detected based on a distribution of the point cloud data; In the second process, a behavior of each person is estimated based on a time change of the point cloud data. The person analysis system according to claim 1 .

3. the first process detects a position of each person using a first neural network that has been trained in advance to output a position where a person is present when the point cloud data is input; the second process estimates the behavior of each person using a second neural network that has been trained in advance to output the behavior of the person when the time change of the point cloud data is input; The person analysis system according to claim 2 .

4. The point cloud data includes at least coordinate information indicating three-dimensional coordinates of each point and motion information indicating a direction and a speed of movement of each point, The second process estimates a behavior of a person by inputting at least the movement information into the second neural network. The person analysis system according to claim 3 .

5. The person analysis device includes: In the first process, a posture and a direction of a person are further detected based on a distribution of the point cloud data; determining whether the person's behavior is appropriate based on the estimated behavior of the person and the detected posture and orientation of the person; displaying the pictogram together with information indicating a judgment result as to whether the person's behavior is appropriate or not. The person analysis system according to any one of claims 2 to 4.

6. The person analysis device includes: Estimating the vital signs of a person based on the point cloud data and the observation data; determining a physical condition of the person based on the estimated behavior of the person, the detected posture of the person, and the estimated vital signs of the person; displaying the pictogram together with information showing the result of the judgment of the person's physical condition. The person analysis system according to any one of claims 2 to 4.

7. the second neural network is configured to output a score indicating the likelihood of estimating the behavior for each type of behavior, The person analysis device includes: When there are a plurality of actions whose scores are equal to or greater than a predetermined threshold, the plurality of pictograms are displayed together. The person analysis system according to claim 3 .

8. The person analysis device includes: displaying the pictogram indicating the behavior with the highest score among the plurality of behaviors in a color or brightness different from the pictograms indicating other behaviors; The person analysis system according to claim 7.

9. the second neural network is configured to output a score indicating the likelihood of estimating the behavior for each type of behavior, The person analysis device includes: when there is no behavior whose score is equal to or greater than a predetermined threshold, displaying one or more of the pictograms from among those with the highest scores in a color or brightness different from that of the pictograms that are displayed when the score is equal to or greater than a predetermined threshold. The person analysis system according to claim 3 .

10. The person analysis device includes: Based on the position of each detected person, the closeness of each person to other people is identified, and information indicating the closeness of each person is displayed together with the pictogram. The person analysis system according to any one of claims 2 to 4.

11. In the information processing device, observation data including a result of observing the monitoring area by a radar system is acquired from at least one radar device installed in the monitoring area; executing a first process of detecting a position of each person present in the monitoring area based on the observation data, and a second process of estimating a behavior of each of the detected people; displaying an area map on a predetermined display device, in which pictograms corresponding to each of the persons present in the surveillance area are arranged, and controlling the display of the pictograms to identify the location and type of behavior of each of the persons; Person analysis method.

12. A method for analyzing a person according to claim 11, Character analysis program.

Images