Information processing device, information processing method, and program

Abstract

To provide an information processor, an information processing method, and a program that allow a user to know an evaluation value at each position within a predetermined area.SOLUTION: An information processor 100 includes: an acquisition unit that acquires an imaged image obtained by imaging an area where an object is active; an object position detection unit that detects, on the basis of the acquired imaged image, a position of the object existing in the area; an action content identification unit that identifies, on the basis of the acquired imaged image, action contents of an object; an attention degree map management unit 505 that manages information where the position within the area and an evaluation values are associated; and an output unit that outputs the information. The attention degree map management unit updates the information according to the position detected by the object position detection unit and the action contents identified by the action content identification unit.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present disclosure relates to processing based on captured images. Background Art

[0002] Methods have been proposed that use camera information to detect a state within a predetermined area.

[0003] Patent Document 1 describes recording movement trajectories of a person moving inside a house with a camera, and estimating a range including the movement trajectories as a soiled area. Prior Art Documents Patent Documents

[0004] Patent Document 1 Japanese Unexamined Patent Application Publication No. 2019-84165 Summary of the Invention Problem to be Solved by the Invention

[0005] A user may desire an operation in which maintenance is performed only on locations within a predetermined area that have a high degree of deteriorated condition. In this case, it is required to detect the degree of deterioration of the condition at each position within the predetermined area. However, the method of uniformly estimating that a range determined based on a person's movement trajectory has the same condition, as in Patent Document 1, cannot detect the degree of deterioration of the condition at each position.

[0006] Additionally, in response to an increase in the prevalence of allergic diseases, there is a demand for detecting positions within a predetermined area that have a high probability of containing allergens. However, the method of uniformly estimating that a range determined based on a person's movement trajectory has the same condition, as in Patent Document 1, cannot detect positions with a high probability of containing allergens. Means for Solving the Problem

[0007] The information processing apparatus of this disclosure includes: acquisition means for acquiring an image obtained by imaging a region in which an object is active; detection means for detecting the position of the object in the region based on the acquired image; and means for identifying the behavior of the object based on the acquired image. First A means for identifying information, a means for managing information relating the position and evaluation value within the said area, and an output means for outputting the said information. A second identification means for identifying objects present in the aforementioned region, The management means is The system acquires allergen data for each object, and if the first object identified by the second identification means possesses the target allergen, it updates the information according to the location where the detection means detected the first object and the behavior of the first object identified by the first identification means. It is characterized by the following: [Effects of the Invention]

[0008] According to this disclosure, the user can know the evaluation value at each position within a given area. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram showing the hardware configuration of an information processing device. [Figure 2] A diagram showing an example of the target area for generating an attention map. [Figure 3] This is a diagram illustrating an example of the installation of an imaging device. [Figure 4] This is a diagram illustrating an example of the installation of an imaging device. [Figure 5] This is a block diagram showing an example of the functional configuration of an information processing device. [Figure 6] This figure shows an example of a table that updates the level of attention. [Figure 7] This is a flowchart illustrating an example of the process for updating attention levels. [Figure 8] A diagram showing an example of an updated popularity map. [Figure 9] This figure shows an example of an updated attention map resulting from link maintenance activities. [Figure 10] This diagram illustrates one example of how to use the popularity map. [Figure 11]FIG. 1 is a diagram for explaining an example of how to use an attention map. [Figure 12] FIG. 2 is a diagram showing an example of a region for which an attention map is to be generated. [Figure 13] FIG. 3 is a diagram showing an example of an attention level update table. [Figure 14] FIG. 4 is a diagram showing an example of an attention map with an updated attention level. [Figure 15] FIG. 5 is a diagram for explaining an example of how to use an attention map. [Figure 16] FIG. 6 is a diagram showing an example of a region for which an attention map is to be generated. [Figure 17] FIG. 7 is a diagram showing an example of an attention map corresponding to an allergen to be managed. [Figure 18] FIG. 8 is a block diagram showing an example of a functional configuration of an information processing device. [Figure 19] FIG. 9 is a diagram showing an example of allergen retention information. [Figure 20] FIG. 10 is a diagram showing an example of an attention level update table. [Figure 21] FIG. 11 is a flowchart for explaining a processing example of attention level update processing. [Figure 22] FIG. 12 is a diagram showing an example of a region for which an attention map is to be generated. [Figure 23] FIG. 13 is a diagram showing an example of an attention map with an updated attention level. [Figure 24] FIG. 14 is a diagram showing an example of managing allergic constitution information for each person. [Figure 25] FIG. 15 is a diagram showing an example of allergen retention information. DETAILED DESCRIPTION OF EMBODIMENTS

[0010] Hereinafter, details of the technology of the present disclosure will be described based on embodiments with reference to the accompanying drawings. Note that the configurations shown in the following embodiments are merely examples, and the technology of the present disclosure is not limited to the illustrated configurations.

[0011] <Embodiment 1> [System Configuration and Hardware Configuration] Figure 1 is a block diagram showing the system configuration of the attention location detection system and the hardware configuration of the information processing device 100 included in the attention location detection system. In this embodiment, an attention location detection system for generating information (map) representing evaluation values ​​(attention level) that indicate the degree to which a user should pay attention to each location in a predetermined area is described.

[0012] The attention area detection system of this embodiment includes an imaging device 110 and an information processing device 100. The imaging device 110 consists of one or more imaging devices. The imaging device 110 captures the target area for generating the attention map, which will be described later, and provides the resulting captured image to the information processing device 100 via IF 104. In this embodiment, the imaging device 110 is described as capturing moving images, but the captured image may be a still image.

[0013] The information processing device 100 has a CPU 101, ROM 102, RAM 103, and IF (interface) 104, each connected by a bus 105.

[0014] The CPU 101 performs operational control to update the attention level, as described later, according to the program stored in the ROM 102 or the program loaded into the RAM 103. The ROM 102 is a read-only memory that stores the boot program, firmware, various processing programs for implementing the processes described later, and various data. The RAM 103 is a work memory where the CPU 101 temporarily stores programs or data for processing, and various processing programs or data are loaded into it by the CPU 101.

[0015] The information processing device 100 may have one or more dedicated hardware components different from the CPU 101, and at least a portion of the processing performed by the CPU 101 may be executed by the dedicated hardware. Examples of dedicated hardware include ASICs (Application-Specific Integrated Circuits), FPGAs (Field-Programmable Gate Arrays), and DSPs (Digital Signal Processors).

[0016] IF104 is an interface for communicating with external devices via a network, and the information processing device 100 can send and receive data via the network.

[0017] At least one of the display unit and the operation unit may be connected via IF104. In this embodiment, at least the display unit 520 (shown in Figure 5) is connected to the information processing device 100. Alternatively, at least one of the display unit 520 and the operation unit (not shown) may be located inside the information processing device 100. The CPU 101 operates as a display control unit that controls the display unit 520 and an operation control unit that controls the operation unit (not shown).

[0018] [Examples of imaging device installation] Figure 2 shows the skating rink 200 where a figure skating competition is held, which is the location where the imaging device 110 of the area of ​​interest detection system of this embodiment performs imaging.

[0019] In sports such as skating and golf, the condition of the field significantly impacts the athletes' performance. There have been cases where severely deteriorated areas of the field have had a major negative impact on athletes' performance.

[0020] In fields containing water, such as ice rinks, the surface of the field reflects light. Furthermore, the actions taken by users on the field significantly impact its quality (condition). Therefore, due to the characteristics of the field, it can be difficult to evaluate its quality from captured images. Areas where the quality of the field has deteriorated (worsened condition) should either be maintained, or users should avoid using the field in such areas.

[0021] Therefore, the attention detection system of this embodiment generates an attention level map that represents the degree to which the user should pay attention to each location within the skating rink. For this reason, the imaging device 110 is installed so that the entirety or part of the skating rink 200 is included in the field of view. In the skating rink 200, a skater 210, who is a user, is skating. Figure 2 illustrates the skater 210 skating, taking off for a toe jump, and landing the jump.

[0022] Figure 3 shows an example of the arrangement of the imaging device 110. Figure 3 shows an example of the installation of the imaging device when there is only one imaging device 110. The imaging device 110 is installed as a zenith camera 300, as shown in Figure 3. The zenith camera 300 is installed, for example, on the ceiling and images the area including the skating rink 200 from the zenith.

[0023] Figure 4 shows an example of the installation of multiple imaging devices, such as video cameras, when the imaging device 110 is composed of multiple imaging devices. Each of the multiple imaging devices 400 is positioned around the skating rink 200. Each imaging device in the multiple imaging devices 400 partially images the skating rink 200, and the multiple imaging devices 400 are positioned so that all the images captured by the multiple imaging devices 400 cover the entire skating rink 200. It is also possible to install a combination of a zenith camera 300 and multiple imaging devices 400.

[0024] [About the Popularity Map] The information processing device 100 of this embodiment generates an attention map representing the level of attention at each location on the ice rink 200. In Figures 3 and 4, for illustrative purposes, the attention map 301 is superimposed on the ice rink 200. The area to which the attention map 301 is generated is the area included in the imaging range of the imaging device 110, and includes the area where users or administrators perform their activities. For example, the area to which the attention map 301 is generated includes the ice rink 200, which is the area where users such as skaters or administrators such as mechanics perform their activities. Note that the area to which the attention map 301 is generated is not limited to the ice rink 200.

[0025] The attention map 301 is divided into rectangular regions 302 of a predetermined size, corresponding to each location within the attention map. Each divided rectangular region 302 is associated with the current attention level of that region. Attention level is described as a value indicating the degree of deterioration of the state of that location (rectangular region). In this embodiment, attention level is also a value indicating the degree of need for maintenance. The higher the attention level, the worse the state of the rectangular region associated with that attention level is, and the higher the degree to which the user should pay attention. In other words, the higher the attention level, the higher the degree to which the rectangular region associated with that attention level needs maintenance. The minimum value of attention level is 0 and there is no limit to the maximum value, but a limit may be placed on the maximum value.

[0026] [Functional Configuration] Figure 5 is a block diagram showing the functional configuration of the information processing device 100. The information processing device 100 includes an acquisition unit 501, an object extraction unit 502, an object position detection unit 503, an action content identification unit 504, an attention level reset unit 506, an attention level update unit 507, and an output unit 508.

[0027] The acquisition unit 501 acquires the captured image obtained by the imaging device 110.

[0028] The object extraction unit 502 extracts pre-specified targets (objects) from the captured image obtained by the imaging device 110. In this embodiment, if the area to be generated for the attention map 301 is the ice rink 200, objects such as skaters, ice rink mechanics, and ice rink maintenance equipment are extracted as objects. One method for extracting objects from the captured image is to perform object detection processing using a trained model generated using deep learning.

[0029] When the imaging device 110 is composed of multiple imaging devices 400 as shown in Figure 4, multiple images obtained by the multiple imaging devices 400 are aggregated in the object extraction unit 502. The object extraction unit 502 then integrates the multiple images and performs a process to extract objects from the integrated image.

[0030] The object position detection unit 503 detects the position of the object extracted by the object extraction unit 502. The object position detection unit 503 detects the position of the object on the attention map. By detecting the position of the object at predetermined time intervals, the trajectory of the object's movement can be detected.

[0031] One method for detecting the position of an object is to identify an object within the imaging range using a camera capable of measuring the distance to the object using infrared light. Then, the point of contact between the identified object and the ground is detected as the object's position. Alternatively, the position in the image captured by the imaging device 110 is associated with the position on the attention map, and the position of the object on the attention map is detected from the position of the object in the image.

[0032] The size of the rectangular area 302 of the attention map 301 depends on the accuracy of the object position detection unit 503's position recognition; the higher the accuracy of the object position detection unit 503's position recognition, the smaller the size of the rectangular area 302 of the attention map 301 can be.

[0033] The action content identification unit 504 identifies the action being performed by the object extracted by the object extraction unit 502 at the location detected by the object location detection unit 503. For example, this process can be realized using deep learning technology. Specifically, supervised machine learning is performed using deep learning, with the input being an image of an object and the output being the action content, and the actual action content shown in the input image as the ground truth data, to generate a trained model. Then, the action content identification unit 504 identifies the action of the object at that location based on the output obtained by inputting the image of the object extracted from the captured image into the generated trained model.

[0034] The attention level update unit 507 determines a value (update value) for updating the attention level corresponding to the behavior identified by the behavior content identification unit 504, using the attention level update table.

[0035] Figure 6 shows an example of an attention update table. The attention update table 600 in Figure 6 associates and stores the actions of an object with a value for updating its attention level. The attention update table 600 in Figure 6 is an attention update table generated when the target area for generating the attention map 301 is the ice rink 200.

[0036] Column 601 of the attention update table 600 holds the behaviors that the behavior identification unit 504 can identify. The attention update table 600 in Figure 6 was generated to update the attention level of the attention map corresponding to the ice rink, and the object behaviors hold the behaviors that objects detected on the ice rink may perform. The behavior identification unit 504 identifies the behaviors of objects using a trained model that has been trained to output the behaviors held in column 601 of the attention update table 600.

[0037] Column 602 of the attention update table 600 holds a value for updating the attention level corresponding to the action content in column 601. In this embodiment, a higher attention level indicates a greater need for maintenance of the skating rink. Therefore, if the action content of an object is an action other than maintenance-related activity, a positive value is associated with it. Furthermore, column 602 holds values ​​such that larger positive values ​​are associated with activities that could cause holes or damage to the rink.

[0038] The attention level update unit 507 updates the attention level of the location detected by the object position detection unit 503 in the attention level map. Specifically, it adds the current attention level value of the object's location (rectangular area) detected by the object position detection unit 503 with the value corresponding to the object's action in Figure 6. The resulting sum is then updated as the attention level of the location (rectangular area) detected by the object position detection unit 503.

[0039] The attention reset unit 506 resets the attention level of the attention map to 0. The attention update table 600 in Figure 6 includes an entry for "link maintenance" as an action, as shown in row 603. "Link maintenance" is a reset action, and the value "reset," which instructs the attention level of the attention map 301 to be set to 0, is associated with the value for updating the attention level. If the value determined by the attention update unit 507 is "reset," the attention reset unit 506 resets the attention level of the attention map associated with the position detected by the object position detection unit 503 to 0. The attention update unit 507 and the attention reset unit 506 are sometimes collectively referred to as the attention map management unit 505.

[0040] The attention level map, after the attention level has been updated, is stored in the memory unit 510 of the attention location detection system. The memory unit 510 in which the attention level map is stored may be a memory unit located outside the information processing device 100, or it may be a memory unit such as the ROM 102 within the information processing device 100.

[0041] The output unit 508 outputs the attention level map stored in the storage unit 510 of the attention detection system so that it can be displayed on the display unit 520 via the IF104.

[0042] Each functional unit within the information processing device 100 in Figure 5 is realized by the CPU 101 of the information processing device 100 executing a predetermined program, but is not limited to this. Other hardware, such as a GPU or FPGA (not shown), may also be used. Each functional unit may be realized through the cooperation of software and hardware such as a dedicated IC, or some or all of the functions may be realized by hardware alone.

[0043] [flowchart] Figure 7 is a flowchart showing the procedure for updating the attention level of the attention map by the information processing device 100. The series of processes shown in the flowchart of Figure 7 are performed by the CPU of the information processing device 100 loading the program code stored in ROM into RAM and executing it. Some or all of the functions of the steps in Figure 7 may also be implemented in hardware such as an ASIC or electronic circuit. The symbol "S" in the description of each process means that it is a step in that flowchart, and the same applies to subsequent flowcharts.

[0044] In S701, the acquisition unit 501 acquires the captured image obtained by the imaging device 110. If the imaging device 110 is capturing a video, for example, one frame's worth of image is acquired. Images from several frames may also be acquired.

[0045] In S702, the object extraction unit 502 extracts objects contained in the captured image acquired in S701. If the object extraction unit 502 is unable to extract objects from the captured image, the process returns to S701, acquires the captured image for the next time point, and steps S701 to S702 are repeated until an object is extracted.

[0046] In S703, the object position detection unit 503 detects the position of the object extracted in S702 on the attention map 301 during imaging.

[0047] In S704, the action content identification unit 504 identifies the action content of the object extracted in S702. The action content is identified from the action content included in the attention update table 600. In S704, the content of the activity that the object extracted in S702 was engaged in at the location detected in S703 is identified.

[0048] In S705, the attention reset unit 506 determines whether the action identified in S704 is a reset action. As mentioned above, the reset action in this embodiment is "link maintenance".

[0049] If the identified action is not determined to be a reset action (S705 is NO), the process proceeds to S706. In S706, the attention update unit 507 obtains the value associated with the action identified in S704 in the attention update table 600 in Figure 6.

[0050] Then, the process proceeds to S707, where the attention level update unit 507 adds the value obtained in S706 to the attention level of the rectangular area in the attention level map that includes the position detected in S703. Then, the attention level of the rectangular area in the attention level map that includes the position detected in S703 is updated to the value after the addition.

[0051] On the other hand, if a reset action is determined (S705 is YES), the process proceeds to S708. In S708, the attention reset unit 506 resets the attention value of the rectangular area in the attention map 301 that includes the position detected in S703. Specifically, it sets the evaluation value associated with the rectangular area in the attention map 301 to 0.

[0052] When S707 or S708 is performed, the attention map 301 is updated to show the maintenance needs for each location on the skating rink 200. The processes S701-708 are repeated until the user gives a command to stop updating the attention map. That is, once S707 or S708 is completed, the process returns to S701. After returning to S701, the image (frame) for the next time is acquired and S701-708 is performed again. On the other hand, if the user gives a command to stop updating the attention map due to maintenance or business closure (S709 is YES), the flowchart ends.

[0053] [Attention map output as a result of processing] Figure 8 shows an example of a attention map generated (updated) by executing the flowchart in Figure 7. The attention map 301 in Figure 8 is an attention map that was updated and generated based on the captured images (frames) obtained by the imaging device 110 at each time point during the time when the skater 210 was skating on the ice rink 200, as shown in Figure 2.

[0054] The numbers contained within the rectangular regions obtained by dividing the attention map 301 in Figure 8 represent the attention level associated with that rectangular region, and these attention levels are updated by repeatedly executing the flowchart in Figure 7. Note that in Figure 8, the attention level values ​​are not shown for rectangular regions with an attention level of 0.

[0055] In the skating rink 200 shown in Figure 2, the skater 210 is assumed to have skated along the skating trajectory 202, taken off for a toe jump at the toe jump takeoff position 203, and landed at the jump landing position 204. The action content identification unit 504 identifies the actions of the skater 210 at each time point in time, based on the images captured by the imaging device 110 at each time point in time during which the skater 210 was skating.

[0056] As shown in Figure 8, the object position detection unit 503 detected the object's position, and as a result, the positions of the skating trajectory 202, the toe jump takeoff position 203, and the jump landing position 204 were detected in the attention map 301. It should be assumed that the attention level before skating was 0 in all rectangular areas.

[0057] In Figure 8, the value "5" associated with "skating" in the attention update table 600 is added to the attention levels of the rectangular regions 801-806 that contain the location of the skating trajectory 202. As a result, the attention map 301 is updated so that the attention level of "5" is directly associated with the rectangular regions 801-806.

[0058] Similarly, the value "80" associated with "toe jump" in the attention update table 600 is added to the attention level of the rectangular area 807 in the attention map that contains the toe jump takeoff position 203. As a result, the attention map 301 is updated so that "80" is directly associated with the attention level of the rectangular area 807.

[0059] Furthermore, the value "40" associated with "jump landing" in the attention update table 600 is added to the attention level of the rectangular area 808 in the attention map that contains the jump landing position 204. As a result, the attention map 301 is updated so that "40" is directly associated with the attention level of the rectangular area 808.

[0060] If the user is a skater using the skating rink 200, the user can use the skating rink 200 while avoiding areas with high attention. If the user is the manager of the skating rink 200, the user can instruct the maintenance staff to perform maintenance on areas where the attention level is higher than a predetermined value. In addition, a maintenance device configured to receive the attention level map 301 may be configured to automatically perform maintenance on areas where the attention level on the attention level map 301 is higher than a threshold.

[0061] Figure 9 shows the attention map 301 output after maintenance was performed on the skating rink 200 in the state shown in the attention map 301 of Figure 8. That is, it is an attention map generated by updating the attention level based on the captured images (frames) obtained by the imaging device 110 at each time point while a maintenance worker or maintenance equipment was performing maintenance on the skating rink 200 in Figure 2. The area 900 enclosed by the dotted line is the area where the position of an object that is a maintenance worker or maintenance equipment was detected, and it indicates the area where that maintenance worker or maintenance equipment was identified as having performed maintenance on the rink. The area 900 includes rectangular areas 805 and 806 which contain the position of the skating trajectory 202, and rectangular area 807 which contains the toe jump takeoff position 203, but the attention level associated with rectangular areas 805 to 807 has been updated to 0.

[0062] Figure 10 is a diagram illustrating a specific example of using the attention map with the display unit 520. Using Figure 10, an example of using the attention map when the user is a user of the skating rink 200 (a skater or coach) will be explained.

[0063] The planned jump takeoff point 1001 indicates the planned jump takeoff point originally planned by the user. The user can check the current level of attention at the planned jump takeoff point 1001 using a map reference device 1000 which has a display unit 520 and can refer to the attention level map 301.

[0064] By checking the attention map 301, users can confirm before starting to skate that the current planned jump takeoff point 1001 is a high-attention, poor-condition area. Therefore, users can change their skating plan in advance, such as changing the planned jump takeoff point to the revised planned jump takeoff point 1002. In this way, according to this embodiment, by referring to the attention map 301, it is possible to suppress the impact of field quality on the athlete's performance.

[0065] Figure 11 illustrates an example of using the attention map 301 when the user is the manager of the skating rink 200. The maintenance areas 1100 and 1101 enclosed by dotted lines are the areas where the user has determined, using the attention map 301, that maintenance will be performed by a maintenance worker or maintenance equipment 1102 and 1103. The user can use the map reference device 1000 to determine the maintenance plan so that the areas containing locations in the attention map 301 where the level of attention is higher than a threshold are designated as the maintenance areas 1100 and 1101.

[0066] As described above, this embodiment allows the maintenance area to be predetermined, enabling users to perform maintenance on holes or scratches on the skating rink caused by jumps, etc., in short periods of time, such as during skater changes in figure skating competitions. Furthermore, after maintenance is performed, users can check for any missed areas by reviewing the current attention map 301. In this way, users can identify areas with high attention as areas requiring focused maintenance and prioritize maintenance on those areas.

[0067] <Embodiment 2> In Embodiment 1, the area for generating the attention map was described as an ice rink. Similar to an ice rink, even in fields with grass, such as soccer fields or golf courses, the field condition cannot be properly evaluated by only considering the visible area. Therefore, in this embodiment, we will describe a point of interest detection system that uses a golf course as the area for generating the attention map. This embodiment will be described mainly in terms of the differences from Embodiment 1. Unless otherwise specified, the configuration and processing are the same as in Embodiment 1.

[0068] Figure 12 shows a golf course 1200, which is the location where the imaging device 110 of the attention detection system of this embodiment performs imaging. The golf course 1200 in Figure 12 is the entire golf course and is the target area for generating the attention map output in this embodiment. The installation position of the imaging device 110 is determined so that the entirety or part of the golf course 1200 is included in the field of view. When covering a wide area such as a golf course, it is preferable to arrange the imaging device 110 in the configuration of multiple imaging devices shown in Figure 4.

[0069] Figure 13 shows an example of an attention update table in this embodiment. The attention update table 1300 in Figure 13 is an attention update table generated when the area for generating the attention map is a golf course. The attention update table 1300 in Figure 13 associates the actions of objects on the golf course with the values ​​used to update their attention.

[0070] The behavior content in column 1301 of the attention update table 1300 in Figure 13 contains the content of activities that an object detected on the golf course 1200 may perform. The behavior content identification unit 504 of this embodiment identifies the behavior content of an object using a trained model that has been trained to output the behavior content held in column 1301 of the attention update table 1300.

[0071] In this embodiment, if the action involves maintenance, a negative value is associated with it as a value for updating the attention level. If the sum of the negative values ​​results in a negative value, the attention level is updated to the minimum value of 0.

[0072] Furthermore, the action details for row 1303 are recorded as "not used for 1 day," and a negative value is associated with it as a value for updating the attention level. The attention level update unit 507 in this embodiment calculates the period during which an object was not located in each rectangular area within the attention level map. When the period during which an object was not located in a rectangular area becomes 1 day, the attention level of that rectangular area is updated by adding the negative value "-10".

[0073] [Attention map output as a result of processing] Figure 14 shows an example of the attention map of this embodiment, generated (updated) by executing the flowchart in Figure 7. Since there is no reset action in the attention update table 1300 of this embodiment, step S705 in Figure 7 can be skipped and the process can proceed to S706. In parallel with the flowchart in Figure 7, the period during which no object was located in each rectangular area constituting the attention map 1410 is calculated, and the attention level is updated according to the calculated period.

[0074] The attention map 1410 in Figure 14 is an attention map generated by updating the attention level based on the captured images (frames) obtained by the imaging device 110 at each time point during the time when users were playing golf at the golf course 1200 in Figure 12.

[0075] In the golf course 1200 shown in Figure 12, it is assumed that the user takes a driver shot at driver shot location 1201 and an iron shot at iron shot location 1202. Furthermore, it is assumed that the user starts their running motion at starting point 1203 on the green and runs along the running path 1205 on the green. Additionally, it is assumed that the user takes a putt at putter shot location 1204 and performs a jump at jump location 1206 on the green.

[0076] The activity identification unit 504 identifies the user's activities at each time point in time during the time the user was playing golf, based on the images captured by the imaging device 110 at each time point. The attention level update unit 507 then determines the values ​​associated with the identified activities using the attention level update table 1300 in Figure 13 and updates the attention level, thereby generating the attention level map 1410 shown in Figure 14.

[0077] In the attention map 1410 of Figure 14, the value "30" associated with "other shots" in the attention update table 1300 is added to the attention level of the rectangular area 1400 that contains the driver shot location 1201. As a result, the attention map 1410 is updated so that "30" is directly associated with the attention level of the rectangular area 1400.

[0078] Similarly, the value "80" associated with "iron shot" in the attention update table 1300 is added to the attention level of the rectangular area 1401 that contains the iron shot location 1202. As a result, the attention map 1410 is updated so that "80" is directly associated with the attention level of the rectangular area 1401.

[0079] Furthermore, the value "20" associated with "running on the green" in the attention update table 1300 is added to the attention levels of the rectangular regions 1402 to 1404, which include the location of the running path 1205 on the green. In addition, the value "30" associated with "other shots" in the attention update table 1300 is added to the attention level of the rectangular region 1404 of the attention map, which includes the putter shot location 1204. The attention level value of rectangular region 1404 before the update is "20". Therefore, the attention update unit 507 adds "30" to the current attention level "20" of rectangular region 1404, and the resulting updated value of "50" is associated with the attention level of rectangular region 1404.

[0080] Then, in the attention update table 1300, the value "70" associated with "jumping on the green" is added to the attention level of the rectangular area 1405 that includes the jump point 1206 on the green. As a result, according to this embodiment 2, it is possible to indicate areas where turf maintenance is needed based on the user's activity location and actions on the golf course.

[0081] Figure 15 illustrates an example of using the attention map 1410 when the user is the manager of a golf course 1200. The hole locations 1500 in Figure 15 show the positions of holes on the green determined by the user after course maintenance. The user, as the manager of the golf course 1200, can determine the hole locations by referring to the attention levels within the green area 1501 on the attention map 1410, thus avoiding areas with high attention levels and deteriorating conditions.

[0082] In this way, by referring to the attention map 1410, it is possible to maintain and construct the golf course while considering the load on the grass and greens that is difficult to detect by directly visually inspecting them. As a result, it becomes possible to maintain the condition of the grass and greens at a high quality.

[0083] <Embodiment 3> In response to the increasing prevalence of allergic diseases, there is a growing demand for detecting the presence of allergens within a specified area. However, it is difficult to visually identify minute substances such as allergens from captured images.

[0084] Therefore, in this embodiment, we will describe an example of detecting locations with a high probability of food allergen presence based on a person's actions. This embodiment will be described mainly in terms of the differences from Embodiment 1. Unless otherwise specified, the configuration and processing are the same as in Embodiment 1.

[0085] [Regarding the areas targeted for generating the attention map] Figure 16 shows the interior of a nursery school, which is the location where the imaging device 110 of the attention detection system in this embodiment performs imaging. In this embodiment, the interior of the nursery school is the area to be targeted for generation of the attention map 1607. For example, if the imaging range is to be the indoors of the nursery school, a zenith camera 1601 can be placed as the imaging device 110.

[0086] In this embodiment, the level of attention is described as representing the degree of possibility of the presence of an allergen. In other words, the level of attention represents the degree to which the allergen is dispersed. The higher the value of the level of attention, the higher the probability that the allergen is present (dispersed) in the rectangular area associated with that level of attention. Therefore, as with the embodiments described above, the level of attention in this embodiment also represents the degree of need for maintenance. The minimum value of the level of attention is 0, and there is no limit to the maximum value.

[0087] There have been reports of allergic symptoms developing after contact with even minute amounts of allergens, and this risk is higher in places where children live in groups, such as daycare centers. Therefore, in order to reduce the risk of contact with allergens, the attention location detection system of this embodiment generates and updates an attention map 1607 so that locations within the daycare center that are likely to contain allergens can be identified.

[0088] Figure 17 is a diagram representing the attention map of this embodiment. There may be multiple types of allergens to be managed. In this case, as shown in Figure 17, an attention map corresponding to each allergen to be managed is generated. If the allergens to be managed are at least chicken eggs, wheat, dairy products, and shellfish, attention maps 1607a to 1607d corresponding to the food allergens of chicken eggs, wheat, dairy products, and shellfish are generated.

[0089] The types of allergens to be managed include, for example, the seven specific raw materials that are considered to cause high severity. In addition, if there are children enrolled in the nursery who have allergic symptoms, the allergens that cause those symptoms may be added to the list of allergens to be managed. In this way, the allergens to be managed can be selected according to the user's needs.

[0090] [Functional Configuration] Figure 18 is a block diagram showing the functional configuration of the information processing device 100. In this embodiment, the information processing device 100 identifies a child who has eaten a food allergen, which is a food that causes allergies, and updates the attention level of the attention level map 1607 based on the child's location and the child's actions.

[0091] The information processing device 100 includes an acquisition unit 501, an object extraction unit 502, an object position detection unit 503, an action content identification unit 504, an attention level reset unit 506, an attention level update unit 1803, a person identification unit 1801, an information update unit 1802, and an output unit 508. Components identical to those in Embodiment 1 are denoted by the same reference numerals and detailed descriptions are omitted.

[0092] The acquisition unit 501 acquires the captured image obtained by the imaging device 110.

[0093] The object extraction unit 502 extracts objects from the captured image obtained by the imaging device 110. In this embodiment, the objects are people such as children or childcare workers.

[0094] The person identification unit 1801 identifies which person the object extracted by the object extraction unit 502 is. In this embodiment, it is described as identifying the name of the person who is the object, but it may also identify the ID assigned to each person. The person identification method can be achieved, for example, by using facial recognition technology, person tracking technology, etc.

[0095] The object position detection unit 503 detects the position of the object extracted by the object extraction unit 502.

[0096] The function of the behavior content identification unit 504 is the same as in Embodiment 1. However, the behavior content identification unit 504 of this embodiment identifies the behavior content of an object from "eating behavior," "gargling," and "hand washing," in addition to the behavior content held in column 2001 of the attention update table 2000 of this embodiment (see Figure 20). When the behavior content identification unit 504 identifies "eating behavior" as the behavior content, it is configured to also identify the content of the meal. Furthermore, "eating behavior" is just one example of an action in which a person holds an allergen, and actions that hold an allergen are not limited to eating behavior. "Gargling" and "hand washing" are just examples of actions that eliminate allergens held by a person, and actions that eliminate allergens are not limited to "gargling" and "hand washing."

[0097] The behavior content identification unit 504 identifies the behavior content using a trained model that has been trained to output one of the following: "eating behavior," "gargling," or "hand washing," in addition to the behavior content stored in column 2001 of the attention update table 2000 (see Figure 20).

[0098] The information update unit 1802 updates the contents of the allergen retention information 1900 (see Figure 19). The updated allergen retention information 1900 is stored in the memory unit 1820 of the area of ​​interest detection system. The allergen retention information 1900 will be described later.

[0099] The attention level update unit 1803 determines a value (update value) for updating the attention level and updates the attention level of the location detected by the object position detection unit 503 in the attention level map 1607. The attention level update unit 1803 updates the attention level taking into account the allergen retention information 1900 (see Figure 19). Details will be described later.

[0100] If the attention level reset unit 506 determines that the update value determined by the attention level update unit 1803 is "reset", it resets the attention level of the attention level map associated with the location detected by the object position detection unit 503 to 0.

[0101] The output unit 508 outputs the attention level map stored in the storage unit 510 of the attention detection system so that it can be displayed on the display unit 520 via the IF104.

[0102] [Regarding allergen retention information] Figure 19 shows a table representing allergen retention information 1900, which is data on allergens held by each object. Allergen retention information 1900 is information used to manage what allergens each person (children and caregivers) in the nursery currently holds, and it associates each person's name with the allergens that the person corresponding to that name currently holds.

[0103] Column 1901 of the allergen retention information 1900 stores the names of individuals that the person identification unit 1801 can identify. The allergen retention information 1900 in Figure 19 was generated when the imaging device 110 targeted the inside of a nursery school, and column 1901 stores the names of children and nursery school teachers enrolled in the nursery school that was targeted for imaging.

[0104] Column 1902 of the allergen retention information 1900 stores the names of allergens currently held by the person whose name is in column 1902. The allergens included in column 1902 are updated as needed by the information update unit 1802. Specifically, if the behavior content identified by the behavior content identification unit 504 is "eating behavior", the information update unit 1802 identifies the row in column 1901 in which the person's name identified by the person identification unit 1801 is included. Then, the information update unit 1802 adds the allergen to column 1902 of the identified row if the food that was the target of the "eating behavior" contains an allergen under management. For example, if the person's name identified by the person identification unit 1801 is "Child B", and the behavior content identification unit 504 identifies that the child engaged in eating behavior involving food containing eggs, then "egg" is newly added to column 1902 of row 1903 in which "Child B" is included.

[0105] Furthermore, if the action content identified by the action content identification unit 504 is "gargling" or "hand washing," the information update unit 1802 identifies a row in which the person's name identified by the person identification unit 1801 is contained in column 1901. Then, the information update unit 1802 deletes the allergen stored in column 1902 in the identified row.

[0106] Furthermore, while the allergens included in column 1902 of the allergen retention information 1900 are updated when eating behavior is identified, information about meals eaten before coming to daycare is not reflected. For this reason, information about the menu of meals eaten before coming to daycare may be obtained from the guardian via IF104. The information processing device 100 may also have a function to update column 1902 of the allergen retention information 1900 so that if the meal eaten before coming to daycare contains an allergen that is subject to management, that allergen is stored there.

[0107] [About the popularity update table] Figure 20 shows an example of the attention update table in this embodiment. In the attention update table 2000 in Figure 20, the actions of a person are associated with a value for updating their attention level.

[0108] In Figure 20, column 2001 of the attention update table 2000 contains the actions that a person may take.

[0109] Column 2002 of the attention update table 2000 holds values ​​for updating the attention level corresponding to the actions in column 2001. In this embodiment, a higher attention level indicates a higher probability of contact with the allergen. Therefore, column 2002 holds values ​​such that actions that are more likely to scatter allergens are associated with larger positive values.

[0110] The attention update table 2000 in Figure 20 includes an entry for "cleaning" as an action, as shown in row 2003. "Cleaning" is the reset action in this embodiment, and if the value determined by the attention update unit 1803 is "reset", the attention reset unit 506 resets the attention level of the attention map associated with the position detected by the object position detection unit 503 to 0.

[0111] [flowchart] Figure 21 is a flowchart showing the processing procedure for updating the attention level of the attention level map and updating the allergen retention information by the information processing device 100.

[0112] S2101 is the same step as S701, and the acquisition unit 501 acquires the captured image obtained by the imaging device 110.

[0113] S2102 is the same step as S702, and the object extraction unit 502 extracts objects included in the captured image acquired in S2101.

[0114] In S2103, the person identification unit 1801 identifies the name of the person of the object (person) extracted in S2102.

[0115] S2104 is the same step as S703, and the object position detection unit 503 detects the position of the object extracted in S2102 in the attention map 1607 at the time of imaging.

[0116] In S2105, the behavior content identification unit 504 identifies the behavior content of the object extracted in S2102. The behavior content identification unit 504 identifies the behavior content of the extracted object from the behavior content included in column 2001 of the attention update table 2000, or from eating behavior, hand washing, and gargling.

[0117] In S2106, if the behavior identified in S2105 is one of the following: eating, hand washing, or gargling, the information update unit 1802 updates the allergen retention information 1900 as described above. Specifically, the information update unit 1802 updates the allergens associated with the person name identified by the person identification unit 1801 in the allergen retention information 1900 based on the identified behavior.

[0118] In S2107, the attention reset unit 506 determines whether the action identified in S2105 is a reset action. As mentioned above, the reset action in this embodiment is "cleaning".

[0119] If the identified action is not determined to be a reset action (S2107 is NO), proceed to S2108.

[0120] In S2108, the attention update unit 1803 identifies the row in the attention update table 2000 in Figure 20 in which the behavior identified in S2105 is contained in column 2001, and obtains the update value contained in column 2002 of the identified row. Furthermore, the attention update unit 507 obtains the name of the allergen associated with the person name identified in S2103 in the allergen retention information 1900 in Figure 19.

[0121] Then, the process proceeds to S2109, where the attention level update unit 1803 updates the attention level of the attention level map corresponding to the allergen name obtained in S2108.

[0122] For example, suppose in S2103 the person's name is identified as "Child A," and in S2105 the action of the object is identified as "sneezing." In this case, the names of the allergens currently held by "Child A," namely "chicken egg" and "wheat," are obtained from the allergen retention information 1900 in Figure 19.

[0123] In this case, the attention level update unit 507 updates the attention level of the attention level map 1607a corresponding to "chicken eggs" and the attention level of the attention level map 1607b corresponding to "wheat" with "40", which is associated with "sneeze" in the attention level update table 2000 in Figure 20. Specifically, it adds "40" to the current attention level of the rectangular area in attention level maps 1607a and 1607b that includes the location detected in S2104. Then, it updates the attention level of the rectangular area in each attention level map that includes the location detected in S2104 with the value after the addition.

[0124] Furthermore, if the person identified in S2103 does not possess an allergen, the attention level on the attention map will not be updated, regardless of the actions identified in S2105.

[0125] On the other hand, if a reset action is determined (S2107 is YES), the process proceeds to S2110. In S2110, the attention reset unit 506 resets the attention value of the rectangular area in the attention maps 1607a to 1607d corresponding to all allergens, including the location detected in S2104.

[0126] When S2109 or S2110 is performed, the attention map 1607 is updated. Once S2109 or S2110 is complete, the process returns to S2101. Returning to S2101, the captured image (frame) for the next time point is acquired, and S2101-2110 are performed. On the other hand, if the user requests that the attention map update be stopped due to maintenance or other reasons (S2111 is YES), the flowchart ends.

[0127] [Attention map output as a result of processing] Figure 22 shows an example of the behavior of children in a nursery school, which is the target area for generating the attention map 1607 of this embodiment. It shows that child 2200, whose name is Child A, vomits at position 2201, and child 2202, whose name is Child B, speaks at position 2203. If the allergen retention information was as shown in Figure 19, then child 2200 (Child A) would have allergens for chicken eggs and wheat, and child 2202 (Child B) would have only the wheat allergen.

[0128] Figure 23 shows an example of a focus map generated (updated) by executing the flowchart in Figure 21. Focus maps 1607a and 1607b in Figure 23 are focus maps generated by updating them based on the captured images (frames) at each time point obtained by the imaging device 110 capturing images of the nursery school in Figure 22.

[0129] The numbers contained within the rectangular regions obtained by dividing the attention maps 1607a and 1607b in Figure 23 represent the attention level associated with each rectangular region, and these attention levels are updated by repeatedly executing the flowchart in Figure 21. Note that in Figure 23, the attention level values ​​are not shown for rectangular regions with an attention level of 0. It is assumed that the attention level in all rectangular regions was 0 before the start of the flowchart in Figure 21.

[0130] Based on the behavior of child 2200 (child A), who had allergens of chicken eggs and wheat, only attention maps 1607a and 1607b, which correspond to the names of the allergens that child 2200 (child A) had, will have their attention levels updated. The attention levels of the other attention maps 1607c to d will not be updated.

[0131] First, the value "80" associated with the behavior "vomiting" of child 2200 (child A), who had allergens for chicken eggs and wheat, is obtained from the attention update table 2000. Then, "80" is added to the attention level of the rectangular area 2301 in the egg attention map 1607a, which includes child 2200's position 2201. Similarly, "80" is added to the attention level of the rectangular area 2311 in the wheat attention map 1607b. As a result, attention maps 1607a and 1607b are updated so that the attention level of rectangular areas 2301 and 2311 in attention maps 1607a and 1607b is directly associated with "80".

[0132] Similarly, based on the behavior of child 2202 (child B), who only possessed the wheat allergen, only the attention map 1607b for the wheat allergen corresponding to the possessed allergen is updated. From the attention update table 2000, the value "10" associated with the behavior "utterance" of child 2202 (child B), who only possessed the wheat allergen, is obtained. Then, "10" is added to the attention of the rectangular region 2312 that contains child 2202's position 2203. As a result, the attention map 1607b is updated so that the rectangular region 2312 in the attention map 1607b is directly associated with the attention value "10".

[0133] If the user of the attention map 1607 is the operator of a daycare center, the user can refer to the attention map 1607 displayed on the display unit 520 to identify areas within the daycare center that are likely to contain allergens. Therefore, the user can operate the daycare center so that areas with a higher attention level than a predetermined value are prioritized for cleaning.

[0134] Figure 24 shows an example of a method for managing allergens that cause allergic reactions in each child enrolled in a nursery school. In the example in Figure 24, a column 2401 that stores allergens that cause allergic reactions in each child has been added to the allergen retention information 1900. That is, information on the allergic constitution of a person corresponding to the person's name in column 1901 is stored in column 2401. The information processing device 100 may also have a function to notify the user of a warning when a child who has an allergic reaction to a specific allergen approaches a location with high attention on the attention map corresponding to that allergen.

[0135] Figure 25 illustrates another example of how retained allergens are managed in the allergen retention information 1900. The behavior content identification unit 504 identifies the behavior of an object using a trained model that has been trained to identify more detailed eating behaviors such as "putting food in the mouth," "touching food with hands," and "spilling food on clothes." In this case, if the behavior content identification unit 504 identifies "putting food in the mouth" as the behavior of an object, it is understood that the allergens contained in that food are retained in the mouth. Also, if the behavior content identification unit 504 identifies "touching food with hands" as the behavior of an object, it is understood that the allergens contained in that food are retained on the fingers. Also, if the behavior content identification unit 504 identifies "spilling food on clothes" as the behavior of an object, it is understood that the allergens contained in that food are retained on the clothes. Therefore, as shown in Figure 25, the allergen retention information 1900 may be managed by associating the location where the allergen is retained, such as the mouth, hands, or clothing.

[0136] As shown in Figure 25, by managing the allergens held by individuals, for example, if the action of an object is identified as "handwashing," the information update unit 1802 can delete only the allergens associated with the hands. This makes it possible to manage allergens held by children or childcare workers in detail.

[0137] As described above, according to this embodiment, it is possible to present areas that are highly likely to contain allergens based on a person's location and actions.

[0138] Furthermore, the attention level of this embodiment is updated to increase with actions that generate droplets or involve contact. For this reason, the attention level of this embodiment can also be used as a value indicating the degree to which minute substances such as viruses or bacteria are present. In addition, although the object to be extracted has been described as a human, it may also be an animal other than a human.

[0139] <Other Embodiments> In Embodiments 1 and 2 described above, the area where sports are played was assumed to be the area for generating the attention map. However, it is possible to use areas other than the area where sports are played as the area for generating the attention map. For example, the garden of one's home or the inside of a building such as one's home may be used as the area for generating the attention map. In this case, an attention update table should be generated so that the attention level increases when the need for cleaning is high.

[0140] This disclosure can also take the form of, for example, a system, a device, a data processing method, a program, or a storage medium. Specifically, it may be applied to a system consisting of multiple devices, or to a device consisting of a single device.

[0141] This disclosure can also be realized by performing the following process: supplying software (programs) that realize the functions of the embodiments described above to a system or device via a network or various storage media, and having the computer (or CPU or MPU) of that system or device read and execute the program.

[0142] <Other> The above-described embodiments include the following configurations.

[0143] (Composition 1) An acquisition means for acquiring an image obtained by imaging the region in which an object is active, A detection means for detecting the position of the object present in the region based on the acquired image, A means for identifying the behavior of the object based on the acquired image, A management means for managing information relating the position and evaluation value within the aforementioned region, It has an output means for outputting the aforementioned information, The aforementioned management means is The information is updated according to the location detected by the detection means and the action content identified by the identification means. An information processing device characterized by the following:

[0144] (Configuration 2) The aforementioned management means is The identifying means obtains a value corresponding to the action content identified, and updates the evaluation value associated with the position of the object detected by the detection means based on the value corresponding to the action content. The information processing device according to configuration 1, characterized by the above.

[0145] (Composition 3) The aforementioned evaluation value represents the degree to which maintenance is necessary.

[0146] An information processing device according to configuration 1 or 2, characterized by the above.

[0147] (Composition 4) The aforementioned evaluation value indicates that a higher value indicates a greater need for maintenance. The information processing apparatus according to configuration 3, characterized by the above.

[0148] (Composition 5) The aforementioned management means is The evaluation value associated with the position of the object detected by the detection means is added to the value corresponding to the action content identified by the identification means, thereby updating the evaluation value in the information. The value corresponding to the aforementioned action is a positive value if the action is something other than maintenance-related activities. The information processing apparatus according to configuration 4, characterized by the features described above.

[0149] (Composition 6) The values ​​corresponding to the aforementioned actions are associated with negative values ​​if the action relates to maintenance. The information processing apparatus according to configuration 5, characterized by the features described herein.

[0150] (Composition 7) The aforementioned management means is If the action identified by the identification means is an action related to maintenance, the evaluation value associated with the position of the object detected by the detection means is reset. An information processing device according to any one of configurations 1 to 6.

[0151] (Composition 8) The system further includes extraction means for extracting the object from the acquired image, The detection means detects the position of the object extracted by the extraction means, The identification means identifies the behavior of the object extracted by the extraction means. An information processing device according to any one of configurations 1 to 7, characterized by the above.

[0152] (Composition 9) The aforementioned information is a map representing the aforementioned region. An information processing device according to any one of configurations 1 to 8.

[0153] (Composition 10) The aforementioned map is divided into sections of a predetermined size range, The detection means detects the area in which the object was located as the position of the object. The information processing apparatus according to configuration 9, characterized by the features described therein.

[0154] (Composition 11) The aforementioned area is a skating rink. The identifying means identifies whether the object's actions include at least jumping and skating. An information processing apparatus according to any one of configurations 1 to 10, characterized by the above.

[0155] (Composition 12) The aforementioned area is a golf course. The aforementioned identifying means identifies, as the actions of the object, at least a shot and actions on the green. An information processing apparatus according to any one of configurations 1 to 10, characterized by the above.

[0156] (Composition 13) The system further comprises a second identification means for identifying objects present in the aforementioned region, The aforementioned management means is Retrieve the allergen data held by each object. If the first object identified by the second identification means contains the target allergen, the information is updated according to the location where the detection means detected the first object and the behavior of the first object identified by the identification means. An information processing apparatus according to any one of configurations 1 to 10, characterized by the above.

[0157] (Composition 14) The aforementioned management means is If the first object contains the target allergen, update the information corresponding to the target allergen. The information processing device according to configuration 13, characterized by the above.

[0158] (Composition 15) The system further includes an update means that updates the allergen data held by the object based on the behavior of the object identified by the identification means. An information processing apparatus according to configuration 13 or 14, characterized by the above.

[0159] (Composition 16) The identification means identifies the contents of the meal of the second object identified by the second identification means based on the captured image, The update means updates the allergen data held by the second object based on the identified dietary content. The information processing device according to configuration 15, characterized by the features described herein.

[0160] (Composition 17) The update means is, The contents of the meal the object was in before it was in the area are retrieved, and the allergen data held by the object is updated. An information processing device according to configuration 15 or 16, characterized by the above.

[0161] (Composition 18) The aforementioned management means is If the action of the first object identified by the identification means is an action that scatters allergens, update the information. An information processing device according to any one of the configurations 13 to 17, characterized by the features described herein.

[0162] (Composition 19) The aforementioned evaluation value indicates that a higher value means a higher degree of the target allergen is present. An information processing device according to any one of the configurations 13 to 18, characterized by the features described herein.

[0163] (Composition 20) The acquisition step involves capturing an image obtained by imaging the region where the object is active, and A detection step of detecting the position of the object present in the region based on the acquired image, A selection step to identify the behavior of the object based on the acquired image, A management step for managing information relating the location and evaluation value within the aforementioned region, The system includes an output step that outputs the aforementioned information, In the aforementioned management step, The information is updated according to the position detected in the detection step and the action content identified in the identification step. An information processing method characterized by the following:

[0164] (Composition 21) A program for causing a computer to function as one of the means of an information processing device described in any one of items 1 to 19. [Explanation of symbols]

[0165] 100 Information Processing Devices 501 Acquisition Department 503 Object position detection unit 504 Action content identification section 507 Popularity Update Section

Claims

1. An acquisition means for acquiring an image obtained by imaging the region in which an object is active, A detection means for detecting the position of the object present in the region based on the acquired image, A first identification means for identifying the behavior of the object based on the acquired image, A management means for managing information relating the position and evaluation value within the aforementioned region, Output means for outputting the aforementioned information, A second identification means for identifying objects present in the aforementioned region, It has, The aforementioned management means is The system acquires allergen data for each object, and if the first object identified by the second identification means possesses the target allergen, it updates the information according to the location where the detection means detected the first object and the behavior of the first object identified by the first identification means. An information processing device characterized by the following:

2. The aforementioned management means is The first identification means obtains a value corresponding to the action content it identifies, and updates the evaluation value associated with the position of the object detected by the detection means based on the value corresponding to the action content. The information processing apparatus according to feature 1.

3. The aforementioned evaluation value represents the degree to which maintenance is necessary. The information processing apparatus according to feature 1.

4. The aforementioned evaluation value indicates that a higher value indicates a greater need for maintenance. The information processing apparatus according to claim 3.

5. The aforementioned management means is The evaluation value associated with the position of the object detected by the detection means is added to the value corresponding to the action content identified by the first identification means, thereby updating the evaluation value in the information. The value corresponding to the aforementioned action is a positive value if the action is something other than maintenance-related activities. The information processing apparatus according to feature 4.

6. The values ​​corresponding to the aforementioned actions are associated with negative values ​​if the action is related to maintenance or improvement activities. The information processing apparatus according to feature 5.

7. The aforementioned management means is If the action identified by the first identification means is an action related to maintenance, the evaluation value associated with the position of the object detected by the detection means is reset. The information processing apparatus according to feature 1.

8. The system further includes extraction means for extracting the object from the acquired image, The detection means detects the position of the object extracted by the extraction means, The first identification means identifies the behavior of the object extracted by the extraction means. The information processing apparatus according to feature 1.

9. The aforementioned information is a map representing the aforementioned region. The information processing apparatus according to feature 1.

10. The aforementioned map is divided into sections of a predetermined size range, The detection means detects the area in which the object was located as the position of the object. The information processing apparatus according to feature 9.

11. The aforementioned management means is If the first object contains the target allergen, update the information corresponding to the target allergen. The information processing apparatus according to feature 1.

12. The system further includes an update means that updates the allergen data held by the object based on the behavior of the object identified by the first identification means. The information processing apparatus according to feature 1.

13. The first identification means identifies the contents of the meal of the second object identified by the second identification means based on the captured image, The update means updates the allergen data held by the second object based on the identified meal content. The information processing apparatus according to feature 12.

14. The update means is, The contents of the meal the object was in before it was in the area are retrieved, and the allergen data held by the object is updated. The information processing apparatus according to feature 12.

15. The aforementioned management means is If the action of the first object identified by the first identification means is an action that scatters allergens, update the information. The information processing apparatus according to feature 1.

16. The aforementioned evaluation value indicates that a higher value means a higher degree of the target allergen is present. The information processing apparatus according to feature 1.

17. An information processing method performed by an information processing device, The acquisition step involves capturing an image obtained by imaging the region where the object is active, and A detection step of detecting the position of the object present in the region based on the acquired image, A first identification step involves identifying the behavior of the object based on the acquired image, A management step for managing information relating the location and evaluation value within the aforementioned region, An output step that outputs the aforementioned information, The process includes a second identification step of identifying objects present in the aforementioned region, In the aforementioned management step, The system acquires allergen data for each object, and if the first object identified in the second identification step possesses the target allergen, it updates the information according to the location of the first object detected in the detection step and the behavior of the first object identified in the first identification step. An information processing method characterized by the following:

18. A program for causing a computer to function as each of the means of the information processing apparatus described in any one of claims 1 to 16.

Images