Information processing device, display method, and program

Abstract

To facilitate grasping a possibility of acquiring an infectious disease at a place of an interest.SOLUTION: An image processing device (10) comprises an image processing unit (110) and a risk information generation unit (120). The image processing unit (110) acquires and processes an image generated by an imaging device (20), that is an image including a plurality of people. As an example, the image processing unit (110) calculates, assuming at least some of the plurality of people as a person of reference, a distance between the person and a person closest to the person (a first distance). The risk information generation unit (120) generates, using the first distance, infectious disease risk information in a region of an interest that is a region in which images are captured by the imaging device (20).SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an image processing device, an image processing method, and a program. [Background technology]

[0002] In recent years, image processing has been used for various purposes. For example, Patent Document 1 describes a system that adjusts the environment in a space so that the reach of droplets from a first subject is outside the breathing zone of a second subject, in which the positions and orientations of the faces of the first subject and the second subject are determined by image processing. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2020 / 044826 Summary of the Invention [Problem to be solved by the invention]

[0004] In order to reduce the risk of contracting an infectious disease, it is important to avoid places where there is a possibility of contracting an infectious disease. However, it is difficult to grasp the possibility of contracting an infectious disease at a given place. One of the objectives of the present invention is to make it easier to grasp the possibility of contracting an infectious disease at a given place. [Means for solving the problem]

[0005] According to the present invention, an image processing means processes an image including a plurality of people to calculate a first distance, which is a distance to a person closest to at least a part of the plurality of people; a risk information generating means for generating infection risk information, which is information regarding a risk of contracting an infectious disease or a safety rate of not contracting an infectious disease in a target area, which is an area included in the image, using the first distance; An image processing apparatus is provided, comprising:

[0006] According to the present invention, a computer image processing for calculating a first distance to a person closest to at least a part of the plurality of people by processing an image including the plurality of people; a risk information generation process for generating infection risk information, which is information regarding a risk of contracting an infectious disease or a safety rate of not contracting an infectious disease in a target area, which is an area included in the image, using the first distance; An image processing method is provided that performs the above steps.

[0007] According to the present invention, a computer includes: an image processing function that processes an image including a plurality of people to calculate a first distance, which is a distance to a person closest to at least a part of the plurality of people; a risk information generation function that generates infection risk information, which is information regarding a risk of contracting an infectious disease or a safety rate of not contracting an infectious disease in a target area that is an area included in the image, using the first distance; A program will be provided to help students develop these skills. Effect of the Invention

[0008] According to the present invention, it becomes easier to understand the possibility of contracting an infectious disease in a target location. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram for explaining a usage environment of an image processing device according to an embodiment. [Diagram 2] FIG. 2 illustrates an example of a functional configuration of an image processing apparatus. [Diagram 3] FIG. 4 is a diagram illustrating an example of information stored in a storage unit. [Figure 4] FIG. 2 is a diagram illustrating an example of a hardware configuration of an image processing apparatus. [Diagram 5] 4 is a flowchart showing a first example of processing performed by the image processing device. [Figure 6]FIG. 6 is a diagram showing an example of a method for calculating a first distance performed in step S20 of FIG. 5. [Figure 7] FIG. 6 is a diagram showing a first example of an image displayed on the display unit in step S40 of FIG. 5. [Figure 8] FIG. 6 is a diagram showing a first example of an image displayed on the display unit in step S40 of FIG. 5. [Figure 9] 13 is a flowchart showing a second example of the process performed by the image processing device. [Figure 10] 13 is a flowchart showing a third example of the process performed by the image processing device. [Figure 11] 13 is a flowchart showing a fourth example of the process performed by the image processing device. [Figure 12] 12 shows an example of a screen displayed on the display unit in step S140 of FIG. [Figure 13] 13 is a flowchart showing a fifth example of processing performed by the image processing device. [Figure 14] 13 is a flowchart showing a sixth example of processing performed by the image processing device. [Figure 15] FIG. 13 is a diagram showing current infection risk information displayed on the display unit. [Figure 16] FIG. 13 is a diagram showing future infection risk information displayed on the display unit. [Figure 17] FIG. 7 is a diagram showing a second example of an image displayed on the display unit in step S40 of FIG. 5. [Figure 18] FIG. 7 is a diagram showing a second example of an image displayed on the display unit in step S40 of FIG. 5. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, the same components are given the same reference numerals and the description will be omitted as appropriate.

[0011] 1 is a diagram for explaining a usage environment of an image processing device 10 according to an embodiment. The image processing device 10 is used together with an imaging device 20.

[0012] The imaging device 20 is, for example, a fixed camera, and repeatedly captures an area where a plurality of people, for example, an unspecified number of people, pass through (hereinafter, referred to as a target area). For this reason, the image generated by the imaging device 20 includes a plurality of people. The frame rate of the image generated by the imaging device 20 is arbitrary, but may be, for example, a frame rate that constitutes a moving image. The imaging device 20 then transmits the generated image to the image processing device 10.

[0013] The image processing device 10 processes the image generated by the imaging device 20 to calculate the distance between people in the target area, i.e., the distance between a certain person (hereinafter referred to as a reference person) and the person closest to that person (hereinafter referred to as a first distance).The image processing device 10 then uses this first distance to generate information regarding the risk of contracting an infectious disease in the target area or the safety rate of not contracting an infectious disease (hereinafter referred to as infection risk information).

[0014] In the example shown in Fig. 1, the image processing device 10 is connected to one imaging device 20. However, the image processing device 10 may be connected to multiple imaging devices 20. In this case, the multiple imaging devices 20 capture images of different target areas. Furthermore, each of the multiple imaging devices 20 transmits an image to the outside in association with information identifying the imaging device 20 (hereinafter referred to as imaging device identification information). In this way, infection risk information can be easily generated for each of multiple target areas, for example, 100 or more locations.

[0015] 2 is a diagram showing an example of a functional configuration of the image processing device 10. The image processing device 10 shown in the figure includes an image processing unit 110 and a risk information generation unit 120.

[0016] The image processing unit 110 acquires and processes an image generated by the imaging device 20, i.e., an image including a plurality of people. As an example, the image processing unit 110 calculates the above-mentioned first distance by regarding at least some of the plurality of people as the above-mentioned reference people. A specific example of a method for calculating the first distance will be described later.

[0017] Furthermore, the image processor 110 performs other processes on the image as necessary, and generates various information.

[0018] When the image processing device 10 is connected to a plurality of image capturing devices 20, the image processing unit 110 acquires an image by linking it to the image capturing device identification information of the image capturing device 20 that generated the image.

[0019] The risk information generating unit 120 uses the first distance to generate infection risk information for the target area that is the subject of imaging by the imaging device 20. As an example, the risk information generating unit 120 judges whether the first distance is equal to or less than a reference value, and generates infection risk information using the judgment result. This reference value is determined based on so-called social distance. Social distance is the physical distance that should be maintained between adjacent people to prevent infection with an infectious disease. The magnitude of the reference value is set based on the main infection route of the target infectious disease. For example, for an infectious disease that is mainly transmitted by droplets, a value of 1.5 m or more and 6 m or less is used as the reference value. For an infectious disease that is mainly transmitted by contact, a value of 50 cm or more and 1.5 m or less is used as the reference value.

[0020] The infection risk information indicates, for example, the risk of contracting an infectious disease in a target area or the safety rate of not contracting an infectious disease. In this case, the infection risk information can be generated from the above-mentioned determination result by, for example, the following method.

[0021] (Method 1) The risk information generating unit 120 calculates the number of combinations of people whose first distance is equal to or less than a reference value for each image, and increases the risk indicated by the infection risk information as this number increases. By using this method, the risk information generating unit 120 can generate infection risk information for each image. (Method 2) The risk information generation unit 120 calculates the occurrence per unit time of combinations of people whose first distance is equal to or less than a reference value, and increases the risk indicated by the infection risk information as this occurrence increases. In this method, the risk information generation unit 120 uses the processing results of multiple images generated at different times. (Method 3) In method 2, the risk information generating section 120 uses the amount of occurrence per unit time and per unit area.

[0022] In each of the above methods, the image processor 110 processes a plurality of temporally consecutive images to calculate the duration during which the first distance remains equal to or smaller than the reference value. The risk information generator 120 may increase the risk indicated by the infection risk information as the duration becomes longer.

[0023] There is also a method for generating infection risk information without using the first distance. For example, the risk information generating unit 120 may calculate the density of people per unit area in the target area, and increase the risk indicated by the infection risk information as the density increases.

[0024] Furthermore, the risk information generating section 120 may use the fact that the first distance has become equal to or smaller than the reference value as infection risk information.

[0025] Furthermore, the image processing unit 110 stores the image generated by the imaging device 20 in the storage unit 150. Here, the image processing unit 110 may store information generated by processing the image in the storage unit 150 in association with the image. In the example shown in the figure, the storage unit 150 is a part of the image processing device 10. However, the storage unit 150 may be an external device to the image processing device 10.

[0026] The image processing unit 110 may generate the above-mentioned information by processing the image stored in the storage unit 150. In this case, the image generated by the imaging device 20 can be temporarily stored in the storage unit 150, and then read out from the storage unit 150 at a desired timing and processed. Regardless of the presence or absence of the storage unit 150, the image processing unit 110 can obtain the image generated by the imaging device 20 from the imaging device 20 and process it in real time.

[0027] The image processing device 10 shown in Fig. 2 further includes a display control unit 130. The display control unit 130 superimposes a display for recognizing a combination of people whose first distance is equal to or less than a reference value, i.e., a combination of a reference person and a person closest to that person, on the image to be processed, and causes the display unit 140 to display the display and the image. The display unit 140 has a display. In the example shown in this figure, the display unit 140 is a part of the image processing device 10, but may be located outside the image processing device 10.

[0028] Fig. 3 is a diagram showing an example of information stored in the storage unit 150. In the example shown in this figure, the storage unit 150 stores an image (referred to as image data in Fig. 3) generated by the imaging device 20 in association with information specifying the date and time when the image was generated (for example, the date and time itself or a frame number). The storage unit 150 also stores the image generated by the imaging device 20 together with information obtained by processing the image (referred to as analysis result in Fig. 3). Note that this analysis result may include infection risk information.

[0029] 4 is a diagram showing an example of the hardware configuration of the image processing device 10. The image processing device 10 includes a bus 1010, a processor 1020, a memory 1030, a storage device 1040, an input / output interface 1050, and a network interface 1060.

[0030] The bus 1010 is a data transmission path for transmitting and receiving data among the processor 1020, memory 1030, storage device 1040, input / output interface 1050, and network interface 1060. However, the method of connecting the processor 1020 and the like to each other is not limited to bus connection.

[0031] The processor 1020 is implemented by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like.

[0032] The memory 1030 is a main storage device realized by a RAM (Random Access Memory) or the like.

[0033] The storage device 1040 is an auxiliary storage device realized by a hard disk drive (HDD), a solid state drive (SSD), a memory card, a read only memory (ROM), or the like. The storage device 1040 stores program modules that realize each function of the image processing device 10 (e.g., the image processing unit 110, the risk information generation unit 120, and the display control unit 130). The processor 1020 loads each of these program modules onto the memory 1030 and executes them, thereby realizing each function corresponding to the program module. The storage device 1040 also functions as a memory unit 150.

[0034] The input / output interface 1050 is an interface for connecting the image processing device 10 to various input / output devices.

[0035] The network interface 1060 is an interface for connecting the image processing device 10 to a network. This network is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network). The method for connecting the network interface 1060 to the network may be a wireless connection or a wired connection. The image processing device 10 may communicate with the imaging device 20 via the network interface 1060.

[0036] 5 is a flowchart showing a first example of processing performed by the image processing device 10. First, the image processing unit 110 of the image processing device 10 acquires an image to be processed (step S10). Then, the image processing unit 110 processes this image and calculates the above-mentioned first distance for each person included in the image (step S20). In this calculation, the image processing unit 110 calculates the first distance using the height and position of the person in the image for which the distance is to be calculated, and the up-down direction of the imaging device 20 that generated the image. At this time, as will be described in detail later, the image processing unit 110 uses a value (hereinafter referred to as a reference height) that is preset as the height of a person.

[0037] Next, the risk information generator 120 generates infection risk information using the first distance generated in step S20. An example of the method for generating infection risk information is as described with reference to Fig. 2 (step S30).

[0038] Then, the display control unit 130 causes the generated infection risk information to be displayed on the display unit 140. At this time, the display control unit 130 may cause the image (which may be a video) used when generating the infection risk information to be displayed together with the infection risk information (step S40). Examples of the image displayed here will be described later.

[0039] Fig. 6 is a diagram showing an example of a method for calculating the first distance performed in step S20 in Fig. 5. The image processing unit 110 identifies a person who serves as a reference. Then, the process shown in this diagram is performed for each of the people located around that person.

[0040] First, the image processing unit 110 calculates the height t of the reference person or people located around that person in the image. Here, this is expressed, for example, in the number of pixels. Next, the image processing unit 110 calculates the distance d from the reference person to the people located around that person in the image. Here, d is expressed in the same unit as t (for example, the number of pixels). Next, the image processing unit 110 calculates d / t, and multiplies this value by the above-mentioned reference height to calculate the distance between the reference person and the people located around that person.

[0041] If there is only one other person around the reference person, the distance calculated for that person becomes the first distance. If there are multiple other people around the reference person, the above distance is calculated for each of these multiple people, and the minimum of these distances becomes the first distance.

[0042] As described above, the reference height is set in advance. This reference height may be changed depending on the location (e.g., country) where the imaging device 20 is installed. For example, the average height of adults in the country where the target imaging device 20 is installed is used as the reference height. As a specific processing example, the storage unit 150 stores information specifying the reference height for each imaging device identification information. Then, the image processing unit 110 acquires the imaging device identification information of the imaging device 20 that generated the image to be processed, and reads out the reference height corresponding to this imaging device identification information from the storage unit 150 and uses it.

[0043] Furthermore, when the image processing unit 110 is able to estimate the attributes (for example, at least one of the sex and age group) of the person for whom the height t is to be calculated by image processing, the image processing unit 110 may change the reference height according to this attribute.

[0044] In most images, distortion inherent to the imaging device 20 that generated the image occurs. When calculating the first distance, the image processing unit 110 preferably performs processing to correct this distortion. The image processing unit 110 performs distortion correction processing according to the position of the person in the image. In general, image distortion is caused by, for example, the optical system (e.g., lens) of the imaging device 20 and the vertical orientation of the imaging device 20 (e.g., angle with respect to the horizontal plane). Therefore, the content of the distortion correction processing according to the position of the person in the image is set according to the optical system (e.g., lens) of the imaging device 20 and the vertical orientation of the imaging device 20.

[0045] In the process described using this figure, if an object whose size is standardized to a certain extent is included in the image, image processing unit 110 may calculate the first distance using the size of this object instead of the person's height.

[0046] Fig. 7 is a diagram showing a first example of an image displayed on the display unit 140 in step S40 of Fig. 5. The display control unit 130 causes the display unit 140 to display the image (which may be a video) used when generating the infection risk information, together with the infection risk information. In the example shown in this figure, the display control unit 130 superimposes a display for recognizing a combination of people whose first distance is equal to or less than a reference value on the image, and then causes the display and image to be displayed on the display unit 140.

[0047] As an example, the display control unit 130 causes the display unit 140 to display a mark indicating the combination of people recognized by the image processing unit 110. The display control unit 130 changes the state of this mark depending on whether or not the first distance is equal to or less than a reference value. More specifically, in the example shown in the figure, the two people constituting the combination of people are surrounded by a circle or an ellipse. The display color of this circle or ellipse and the state of the line (solid line, dotted line, dashed line, etc.) change depending on whether or not the first distance is equal to or less than a reference value.

[0048] When the displayed image is a video, the combination of people for which the first distance is calculated changes over time, as shown in Figures 7 and 8. For example, at the timing in Figure 7, person P1 is the person P2's counterpart when calculating the first distance, but at the later timing in Figure 8, person P4 is the person P2's counterpart when calculating the first distance.

[0049] 17 and 18 are diagrams showing a second example of an image displayed on the display unit 140 in step S40 in FIG. 5. In these diagrams, the display control unit 130 displays, for each person, a mark indicating the range of the recommended social distance value (for example, the above-mentioned reference value) centered on that person. Then, when a mark corresponding to a certain person overlaps with a mark corresponding to a person nearby, that is, when the distance between a certain person and a person nearby falls below the recommended social distance value (for example, persons P1 and P2 in FIG. 17 and FIG. 18), the marks corresponding to these two people are overlapped with a mark corresponding to another person (for example, person P3 in FIG. 17, persons P3 and P4 in FIG. 18). 4,The mark is displayed in a different manner from the mark in the previous display mode (P5, P6). The manner can be changed, for example, by changing the display color or by changing the manner of the lines constituting the mark (solid line, dotted line, dashed line, etc.). When changing the display color, the display control unit 130 displays the mark in a normal state in blue, and when two marks overlap, displays these two marks in red.

[0050] In the example shown in Fig. 17, the display control unit 130 overlays the above-mentioned mark on the image (or video) used when generating the infection risk information. On the other hand, in the example shown in Fig. 18, the placement of people is shown in a plan view, and the above-mentioned mark is overlaid on the plan view. The display control unit 130 may cause the display shown in Fig. 17 and the display shown in Fig. 18 to be displayed simultaneously on the display unit 140.

[0051] The displays shown in Figures 7, 8, 17, and 18 may be performed using real-time videos or images. In this case, the displays shown therein may be displayed on a display unit 140 installed near the target area, for example, or may be used as Internet or broadcast content.

[0052] Fig. 9 is a flowchart showing a second example of the process performed by the image processing device 10. The example shown in this figure is similar to the example shown in Fig. 5, except that the image processing unit 110 calculates the second distance when calculating the first distance (step S22), and further uses this second distance to generate infection risk information (step S30).

[0053] The second distance is the distance between the reference person and the second closest person. The method of calculating the second distance is the same as the method of calculating the first distance, except that the distance to the second closest person is selected instead of the first closest person. When generating infection risk information, the risk information generating unit 120 sets the risk to increase (the safety factor decreases) as the second distance becomes closer. Note that the image processing unit 110 may further generate a distance (third distance) between the reference person and the third closest person from that person. In this case, the risk information generating unit 120 further generates infection risk information using the third distance.

[0054] Fig. 10 is a flowchart showing a third example of processing performed by the image processing device 10. The example shown in this figure is similar to the example shown in Fig. 5 or 9, except that the risk information generation unit 120 further uses information other than the distance between people when generating infection risk information.

[0055] Specifically, steps S10 and S20 (or step S22) are similar to the example shown in FIG. 5 (or FIG. 9). Then, the image processing unit 110 processes the image to generate additional information required to generate infection risk information. The information generated here is at least one of the results of identifying the direction of the person's face, the presence or absence and type of an attachment to the face, and the results of identifying the movement of the person's mouth (step S24).

[0056] The "direction of a person's face" includes at least one of the direction of the face of a reference person and the direction of the face of a person closest to that person. The risk information generating unit 120 increases the risk indicated by the infection risk information (decreases the safety factor) as the person's face approaches the direction facing the other person. Here, when the image processing unit 110 and the risk information generating unit 120 use the second distance or the third distance, they may further use the direction of the other person's face when calculating the second distance or the direction of the other person's face when calculating the third distance.

[0057] The "presence or absence of an article on the face" includes at least one of the presence or absence of an article on the reference person and the presence or absence of an article on the person closest to that person. When a specific type of article is detected, the risk information generating unit 120 lowers the risk indicated by the infection risk information (increases the safety factor) compared to other cases. Here, the specific type of article is an article that covers at least one of the mouth and nose (preferably both), such as a mask or a scarf. Here, when the image processing unit 110 and the risk information generating unit 120 use the second distance or the third distance, they may also do the same for the person who is the other party when calculating the second distance and the person who is the other party when calculating the third distance.

[0058] "Mouth movement" means that at least the mouth is moving. If the mouth is moving, there is a high possibility that the person is talking. Therefore, when the mouth of at least one of the reference person and the person closest to the person is moving, the risk information generation unit 120 increases the risk indicated by the infection risk information (decreases the safety factor) compared to other cases. Here, when the image processing unit 110 and the risk information generation unit 120 use the second distance or the third distance, they may further use the mouth movement of the other person when calculating the second distance or the mouth movement of the other person when calculating the third distance.

[0059] 11 is a flowchart showing a fourth example of the process performed by the image processing device 10. The image processing device 10 also performs the process shown in this figure in addition to the process shown in FIG.

[0060] In the example shown in the figure, the imaging device 20 is a fixed camera. Therefore, each position in the image corresponds to a specific position in the target area. The image processing unit 110 stores this correspondence in advance. Then, the image processing unit 110 identifies a location where a proximity index, which is an index regarding the proximity state of multiple people, meets a criterion (hereinafter, referred to as a cautionary location). Then, the risk information generation unit 120 (an example of an output unit) outputs information indicating the cautionary location.

[0061] 5 and 10, or step S22 in Fig. 9, the image processing unit 110 of the image processing device 10 specifies a position in the image where the first distance is equal to or smaller than the reference value, thereby specifying a position in the target region where the first distance is equal to or smaller than the reference value. Then, the image processing unit 110 stores information indicating the position in the storage unit 150 in association with the processed image.

[0062] The risk information generating unit 120 of the image processing device 10 then processes the information stored in the storage unit 150 to count the number of times the first distance was equal to or less than the reference value during the target period (an example of the above-mentioned proximity index) for each position within the target area. The length of the target period may be, for example, one day, one week, or one month.

[0063] In detail, first, the risk information generating unit 120 acquires information for identifying the target imaging device 20 and information for identifying the start and end of the target period. This acquisition is performed, for example, by input from the user. Next, the risk information generating unit 120 reads out the analysis results of the images generated by the target imaging device 20 during the target period from the storage unit 150. The information read out here includes information indicating the position where the first distance is equal to or smaller than the reference value. This information is generated for each image (step S110).

[0064] The target area is divided into a plurality of parts in advance. The risk information generating unit 120 counts the number of times the first distance is equal to or smaller than the reference value for each of the plurality of parts (step S120). Note that the above-mentioned "position where the first distance is equal to or smaller than the reference value" among the information stored in the storage unit 150 may be information indicating this part.

[0065] The risk information generating unit 120 then outputs information indicating the portion where the count number is equal to or greater than the reference value. The portion indicated by this information is an area where there is a high risk of infection, i.e., a place requiring caution. In the example shown in the figure, the risk information generating unit 120 outputs information indicating this portion. This information is output to, for example, the display control unit 130. In this case, the display control unit 130 generates display information for displaying the information indicating this portion (step S130), and causes the display unit 140 to display the display information (step S140).

[0066] 2, image processing unit 110 can calculate the time during which the state in which the first distance is equal to or less than the reference value continues by processing a plurality of temporally consecutive images. In this case, image processing unit 110 may count the number of times the state in which the first distance is equal to or less than the reference value continues for equal to or more than the reference time, and perform the above-mentioned processing.

[0067] Furthermore, the output from the risk information generating unit 120 is used to identify places within the target area where people tend to gather and to improve such places (for example, to improve people's flow lines). As an example, if the target area is indoors, this output is used as reference data for changing the arrangement and number of objects (for example, benches in a waiting room) placed within the room (for example, a waiting room or a hallway). Note that examples of flow lines to be improved include the flow lines from the waiting room to the examination room in a hospital, and the flow lines from the hospital entrance to the treatment room.

[0068] Examples of indoor locations include facilities such as hospitals, government offices, stations, and airports, but may also be large stores such as shopping malls (including those attached to airports and stations). In the latter case, the output from the risk information generating unit 120 identifies locations within the large store where people are likely to gather. The results of this identification are used as reference data when changing the layout and traffic patterns of tenants to prevent people from gathering in those locations.

[0069] In the above example, the unit for setting the "cautionary location" is set by dividing one target area into multiple parts. On the other hand, multiple imaging devices 20 may be connected to the image processing device 10, and these multiple imaging devices 20 may capture different locations in the same facility. In this case, the unit for setting the "cautionary location" may be the imaging area of ​​one imaging device 20 (i.e., one target area). To do this, the image processing unit 110 may count the number of times the first distance becomes equal to or less than the reference value, or the number of times this state continues for a reference time or more, for each imaging device 20, not for each of the multiple parts described above.

[0070] Fig. 12 shows an example of a screen displayed on display unit 140 in step S140 of Fig. 11. In the example shown in this figure, display unit 140 displays a plan view of the target area. Then, display control unit 130 causes display unit 140 to display a portion of this plan view where the count number in step S120 is equal to or greater than the reference value in such a manner that it is distinguishable from other portions.

[0071] Note that multiple values ​​may be set in stages as the reference value for the count number. In this case, the risk information generating unit 120 identifies which of the reference values ​​the count number in step S120 exceeded, and outputs information indicating the identified value. For example, the display control unit 130 may change the display mode of the relevant portion in response to this output. For example, the portion exceeding only the lowest reference value may be displayed in green, and the portion exceeding the highest reference value may be displayed in red.

[0072] Fig. 13 is a flowchart showing a fifth example of the processing performed by the image processing device 10. The image processing device 10 performs the processing shown in this figure in addition to the processing shown in Fig. 5, Fig. 9, or Fig. 10. The image processing device 10 may also perform the processing shown in Fig. 11.

[0073] In this diagram, the risk information generating unit 120 of the image processing device 10 identifies the timing at which the risk of contracting an infectious disease increases (the timing at which the safety margin decreases). This timing is set, for example, by day of the week, by time period, or by day of the week and time period.

[0074] In the example shown in this figure, the process shown in step S110 is the same as the process described with reference to FIG.

[0075] Next, the risk information generating unit 120 counts the number of times that the first distance is equal to or smaller than the reference value for each timing (step S122). Here, the risk information generating unit 120 may further count for each timing for multiple parts in the target area as shown in FIG.

[0076] Next, the risk information generating unit 120 outputs information indicating the timing when the count number exceeded the reference value. The timing (e.g., time period or day of the week) indicated by this information is a timing when the risk of infection is high. As an example, the risk information generating unit 120 outputs information indicating this timing to the display control unit 130. The display control unit 130 then generates display information for displaying the information indicating this timing (step S130), and causes the display information to be displayed on the display unit 140 (step S140).

[0077] When the target area is inside a store, the display unit 140 may be provided at the entrance or in a show window of the store. In this way, people who are going to enter the store can know when the store is likely to be empty. The display control unit 130 may also publish the display information generated in step S130 on the Internet. In this way, people who are going to the store can know when the store is likely to be empty.

[0078] Furthermore, when the target area is inside a store and the display unit 140 is installed at the entrance or in a show window of the store, the display control unit 130 may cause the display unit 140 to display current infection risk information, or may cause the display shown in Figures 7, 8, 17, and 18 to be performed using real-time video and images.

[0079] Fig. 14 is a flowchart showing a sixth example of the processing performed by the image processing device 10. The image processing device 10 performs the processing shown in this figure in addition to the processing shown in Fig. 5, Fig. 9, or Fig. 10. The image processing device 10 may further perform at least one of the processing shown in Fig. 11 and Fig. 13.

[0080] In the example shown in the figure, the risk information generation unit 120 stores at least one of the history of past infection risk information and the results of statistical processing of the history in the storage unit 150 for each of the multiple imaging devices 20, i.e., for each of the multiple target areas. Note that these multiple target areas are related to each other in terms of the flow of people. As an example, these multiple target areas may be adjacent to each other, or may be along the same road or railway (subway).

[0081] If the storage unit 150 stores only the history of past infection risk information, the risk information generation unit 120 performs statistical processing on this history. Then, using the result of the statistical processing of the history of infection risk information and the current infection risk information, an estimation result of infection risk information at a predetermined time in the future is generated. The predetermined time may be, for example, one hour later, three hours later, or five hours later, but is preferably within 24 hours. The result of the statistical processing of the history is, for example, a model generated by machine learning, but is not limited to this. Note that this model outputs an estimation result of infection risk information at a predetermined time in the future when the current infection risk information is input.

[0082] First, the risk information generator 120 processes current images generated by each of the imaging devices 20 installed in different target areas to generate current infection risk information for each of the multiple target areas (step S210). At this time, the risk information generator 120 may further process images generated from the present until a predetermined time ago.

[0083] Next, the risk information generation unit 120 acquires the results of statistical processing of the history of infection risk information for each of these multiple target areas. This acquisition may be performed by reading from the storage unit 150, or by performing statistical processing on the history of infection risk information on the spot. Next, the risk information generation unit 120 generates an estimate of infection risk information at a predetermined time in the future using the results of the statistical processing of the history of infection risk information and the current infection risk information (step S220).

[0084] Next, the display control unit 130 generates display information for displaying the current infection risk information and the estimated result of infection risk information at a predetermined time in the future, generated by the risk information generation unit 120 (step S230). The display unit 140 uses this display information to display the current infection risk information and the estimated result of infection risk information at a predetermined time in the future (step S240). A person who sees this display can choose the timing of their action (for example, the timing of getting on a train or the timing of going to a destination) to be a timing with low infection risk.

[0085] When generating an estimate of infection risk information at a predetermined time in the future, the risk information generating unit 120 may identify an increase / decrease trend of the current infection risk information and use this identification result instead of statistical processing. The risk information generating unit 120 identifies this increase / decrease trend, for example, by using the transition in the number of times that the first distance from the past to the present has been equal to or less than a reference value.

[0086] FIG. 15 shows the current infection risk information displayed on the display unit 140. As described above, the risk information generation unit 120 generates infection risk information for each of a plurality of target areas. In the example shown in this figure, the plurality of target areas are areas obtained by dividing one large region into a plurality of areas. The display control unit 130 then causes the display unit 140 to display the target areas in which the risk indicated by the infection risk information has reached a reference value or higher in a state in which the target areas can be distinguished from the other target areas. Note that in this process, the display control unit 130 may also display the progress of the infection risk information from the past to the present.

[0087] FIG. 16 shows future infection risk information displayed on the display unit 140. In the example shown in this figure, the display control unit 130 causes the display unit 140 to display target areas in which the risk indicated by the future infection risk information has reached a reference value or higher in a state in which the target areas can be distinguished from other target areas. Here, if multiple target areas are related to each other in terms of people flow, current infection risk information around a target area can also be used when generating future infection risk information for the target area. This is because people in one target area may move to another target area in a few hours.

[0088] It should be noted that the displays shown in Figs. 15 and 16 may be published on the Internet, or may be included in the content of a television broadcast.

[0089] As described above, according to this embodiment, the image processing unit 110 of the image processing device 10 acquires and processes an image generated by the imaging device 20, i.e., an image including multiple people, to calculate the distance (first distance) to the closest person for at least some of the multiple people. Then, the risk information generation unit 120 uses the first distance to generate infection risk information for the target area that is the subject of image capture by the imaging device 20. This makes it easy to grasp the risk of contracting an infectious disease in the target area.

[0090] In addition, the risk information generating unit 120 outputs a location where a proximity index, which is an index relating to the proximity state of multiple people, meets a criterion. This output is displayed, for example, on the display unit 140. This makes it easier to grasp locations where there is a possibility of contracting an infectious disease.

[0091] Although the embodiments of the present invention have been described above with reference to the drawings, these are merely examples of the present invention, and various configurations other than those described above can also be adopted.

[0092] In addition, in the flowcharts used in the above description, multiple steps (processes) are described in order, but the order of execution of the steps performed in each embodiment is not limited to the order described. In each embodiment, the order of the steps shown in the figures can be changed to the extent that the content is not affected. In addition, the above-mentioned embodiments can be combined to the extent that the content is not contradictory.

[0093] A part or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes. 1. An image processing means for processing an image including a plurality of people to calculate a first distance, which is a distance to a person closest to at least a part of the plurality of people; a risk information generating means for generating infection risk information, which is information regarding a risk of contracting an infectious disease or a safety rate of not contracting an infectious disease in a target area, which is an area included in the image, using the first distance; An image processing device comprising: 2. In the image processing device according to the above item 1, The image processing means further calculates a second distance, which is a distance to a person second closest to the person, The risk information generation means is an image processing device that further generates the infection risk information using the second distance. 3. In the image processing device according to the above 1 or 2, The image processing means further identifies at least one of a face orientation of the person and a face orientation of the closest person; The risk information generation means is an image processing device that further generates the infection risk information using the result of identifying the face direction. 4. In the image processing device according to any one of the above items 1 to 3, The image processing means further identifies at least one of an attachment to the face of the person and an attachment to the face of the closest person; The risk information generation means is an image processing device which further generates the infection risk information using the identification result of the attachment. 5. In the image processing device according to any one of the above items 1 to 4, The image processing means further identifies mouth movements of either the person or the closest person; The risk information generation means is an image processing device that further generates the infection risk information using the results of identifying the mouth movements. 6. In the image processing device according to any one of 1 to 5 above, The image processing means is an image processing device that calculates the first distance using the height and position of the person in the image. 7. In the image processing device according to claim 6, The image processing means includes: Furthermore, the first distance is calculated using a reference height that is a preset value, An image processing device that sets the reference height using the location where the image was generated. 8. In the image processing device according to any one of 1 to 7 above, The risk information generation means is an image processing device that determines whether the first distance is equal to or less than a reference value, and generates the infection risk information using the determination result. 9. In the image processing device according to claim 8, An image processing device comprising a first display control means for superimposing a display on the image to allow the user to recognize the combination of the person and the closest person whose first distance is less than a reference value, and then displaying the display and the image on a display means. 10. In the image processing device according to the above 8 or 9, The image processing means processes a plurality of images of the target area photographed at a plurality of times, The risk information generating means counting the number of times that the first distance is equal to or smaller than a reference value in each of a plurality of portions included in the target region; The image processing device outputs information indicating the portion where the count result satisfies a criterion. 11. In the image processing device according to any one of claims 1 to 10, The image processing means processes a plurality of images of the target area photographed at a plurality of times, The risk information generating means An image processing device that generates the infection risk information for each of the multiple timings, and identifies a timing when the risk increases or the safety factor decreases using the multiple infection risk information. 12. In the image processing device according to any one of claims 1 to 11, The risk information generating means An image processing device that generates an estimation result of the infection risk information at a predetermined time in the future using the result of statistical processing of the history of the past infection risk information and the current infection risk information. 13. In the image processing device according to any one of claims 1 to 12, The image processing means processes the image generated in each of the plurality of target regions to generate the first distance for each of the plurality of target regions; The risk information generation means generates the infection risk information for each of the plurality of target areas, The image processing device further comprises a second display control means for causing a display means to display a target area in which the risk indicated by the infection risk information has reached a reference value or higher in a state in which the target area can be distinguished from other target areas. 14. The computer image processing for calculating a first distance to a person closest to at least a part of the plurality of people by processing an image including the plurality of people; a risk information generation process for generating infection risk information, which is information regarding a risk of contracting an infectious disease or a safety rate of not contracting an infectious disease in a target area, which is an area included in the image, using the first distance; An image processing method. 15. The image processing method according to claim 14, In the image processing, the computer further calculates a second distance which is a distance to a person second closest to the person, An image processing method in which, in the risk information generation process, the computer further generates the infection risk information using the second distance. 16. In the image processing method according to 14 or 15 above, In the image processing, the computer further identifies at least one of a face orientation of the person and a face orientation of the closest person; An image processing method in which, in the risk information generation process, the computer further generates the infection risk information using the results of identifying the face direction. 17. In the image processing method according to any one of claims 14 to 16, In the image processing, the computer further identifies at least one of an attachment to the face of the person and an attachment to the face of the closest person; An image processing method in which, in the risk information generation process, the computer further generates the infection risk information using the results of identifying the attachment. 18. In the image processing method according to any one of the above items 14 to 17, In the image processing, the computer further identifies mouth movements of either the person or the closest person; An image processing method in which, in the risk information generation process, the computer further generates the infection risk information using the results of identifying the mouth movement. 19. In the image processing method according to any one of claims 14 to 18, An image processing method in which, in the image processing, the computer calculates the first distance using the height and position of the person in the image. 20. In the image processing method according to claim 19, In the image processing, the computer Furthermore, the first distance is calculated using a reference height that is a preset value, An image processing method for setting the reference height using the location where the image was generated. 21. In the image processing method according to any one of the above items 14 to 20, An image processing method in which, in the risk information generation process, the computer determines whether the first distance is below a reference value and generates the infection risk information using the determination result. 22. In the image processing method according to claim 21, The computer includes: An image processing method that performs a first display control in which a display for allowing a user to recognize a combination of the person and the closest person whose first distance is less than a reference value is overlaid on the image, and the display and the image are displayed on a display means. 23. In the image processing method according to 21 or 22 above, In the image processing, the computer processes a plurality of images of the target area captured at a plurality of times, In the risk information generation process, the computer counting the number of times that the first distance is equal to or smaller than a reference value in each of a plurality of portions included in the target region; and outputting information indicating the portion where the count result satisfies a criterion. 24. In the image processing method according to any one of claims 14 to 23, In the image processing, the computer processes a plurality of images of the target area captured at a plurality of times, In the risk information generation process, the computer An image processing method comprising: generating the infection risk information for each of the plurality of timings; and using the plurality of infection risk information to identify a timing at which the risk increases or a timing at which the safety factor decreases. 25. In the image processing method according to any one of claims 14 to 24, In the risk information generation process, the computer An image processing method for generating an estimate of infection risk information at a predetermined time in the future using the results of statistical processing of the history of past infection risk information and the current infection risk information. 26. In the image processing method according to any one of the above items 14 to 25, In the image processing, the computer processes the image generated in each of the plurality of target regions to generate the first distance for each of the plurality of target regions; In the risk information generation process, the computer generates the infection risk information for each of the plurality of target areas, Furthermore, the image processing method includes a second display control in which the computer displays, on a display means, target areas in which the risk indicated by the infection risk information has reached a standard value or above in a manner that makes them distinguishable from other target areas. 27. To the computer, An image function that processes an image including a plurality of people to calculate a first distance for at least a portion of the plurality of people, the first distance being a distance to a person closest to the person; a risk information generation function that generates infection risk information, which is information regarding a risk of contracting an infectious disease or a safety rate of not contracting an infectious disease in a target area that is an area included in the image, using the first distance; A program that allows you to have 28. In the program according to 27 above, The image processing function further calculates a second distance to the person closest to the person; The risk information generation function further includes a program that generates the infection risk information using the second distance. 29. In the program according to 27 or 28 above, The image processing function further determines at least one of a facial orientation of the person and a facial orientation of the closest person; The risk information generation function further includes a program for generating the infection risk information using the results of identifying the face direction. 30. In the program according to any one of the above 27 to 29, The image processing function further identifies at least one of an attachment to the face of the person and an attachment to the face of the closest person; The risk information generation function further includes a program for generating the infection risk information using the results of identifying the attachment. 31. In the program according to any one of the above 27 to 30, The image processing function further comprises determining mouth movements of either the person or the closest person; The risk information generation function further includes a program for generating the infection risk information using the results of identifying the mouth movements. 32. In the program according to any one of the above 27 to 31, The image processing function is a program that calculates the first distance using the height and position of the person in the image. 33. In the program according to 32 above, The image processing function includes: Furthermore, the first distance is calculated using a reference height that is a preset value, A program for setting the reference height using the location where the image was generated. 34. In the program according to any one of the above 27 to 33, The risk information generation function is a program that determines whether the first distance is below a standard value and generates the infection risk information using the determination result. 35. In the program according to 34 above, A program that gives the computer a first display control function that overlays a display onto the image for recognizing a combination of the person and the closest person whose first distance is less than a reference value, and then displays the display and the image on a display means. 36. In the program according to 34 or 35 above, The image processing function processes a plurality of images of the target area captured at a plurality of times, The risk information generation function includes: counting the number of times that the first distance is equal to or smaller than a reference value in each of a plurality of portions included in the target region; and outputting information indicating the portion where the count result satisfies a criterion. 37. In the program according to any one of the above 27 to 36, The image processing function processes a plurality of images of the target area captured at a plurality of times, The risk information generation function includes: A program that generates the infection risk information for each of the multiple timings, and uses the multiple infection risk information to identify timings when the risk increases or the safety rate decreases. 38. In the program according to any one of claims 27 to 37, The risk information generation function includes: A program that generates an estimate of the infection risk information at a specified time in the future using the results of statistical processing of the past history of the infection risk information and the current infection risk information. 39. In the program according to any one of claims 27 to 38, The image processing function processes the image generated in each of the plurality of target regions to generate the first distance for each of the plurality of target regions; The risk information generation function generates the infection risk information for each of the plurality of target areas, Further, the program provides the computer with a second display control function for causing a display means to display a target area in which the risk indicated by the infection risk information has reached a reference value or above in a manner that makes it identifiable from other target areas.

[0094] This application claims priority based on Japanese Patent Application No. 2020-098401, filed on June 5, 2020, the disclosure of which is incorporated herein in its entirety. [Explanation of symbols]

[0095] 10 Image processing device 20 Imaging device 110 Image processing section 120 Risk Information Generation Department 130 Display control unit 140 Display section 150 Storage section

Claims

1. a distance acquisition means for acquiring a distance between a first person appearing in an image captured by a camera and another person appearing in the image; an image processing means for displaying, on a display device, a display element visually indicating a reference distance in a manner superimposed on the first person in the image; Equipped with the image processing means changes a display mode of the display element based on a first distance, which is a distance acquired for a second person closest to the first person; Information processing device.

2. The information processing apparatus according to claim 1 , wherein the image processing means changes a display mode of the display element based on the first distance and the reference distance.

3. The information processing apparatus according to claim 2 , wherein the image processing means changes a display mode of the display element based on a magnitude relationship between the first distance and the reference distance.

4. The information processing apparatus according to claim 3 , wherein the image processing means changes a display mode of the display element when the first distance is equal to or less than the reference distance.

5. the image processing means displays the display element in a superimposed manner near the first person based on a position of the first person who is moving; The information processing device according to any one of claims 1 to 4.

6. The apparatus further includes a risk information generating means for generating infection risk information, which is information regarding the risk of infection or the safety rate of infection in the target area included in the image, based on the distance acquired by the distance acquiring means. The information processing device according to any one of claims 1 to 5.

7. The risk information generating means Identifying at least one of a facial orientation of the first person and a facial orientation of the second person; generating the infection risk information using a result of identifying the face orientation; The information processing device according to claim 6.

8. The risk information generating means Identifying at least one of a face attachment of the first person and a face attachment of the second person; and generating the infection risk information by further using the identification result of the wearable item.

8. The information processing device according to claim 6 or 7.

9. The risk information generating means Identifying at least one of a mouth movement of the first person and a mouth movement of the second person; generating the infection risk information using further the results of the mouth movement identification; The information processing device according to any one of claims 6 to 8.

10. the image processing means changes at least one of a color, a shape, and a size of the range of the reference distance as the display mode. The information processing device according to any one of claims 1 to 9.

11. the distance acquisition means calculates a distance between the first person and the other person using a height and a position of the first person; The information processing device according to any one of claims 1 to 10.

12. The computer Obtaining a distance between a first person appearing in an image captured by a camera and another person appearing in the image; displaying a display element visually indicating a reference distance on a display device in a manner superimposed on the first person in the image; changing a display mode of the display element based on a first distance, which is a distance acquired for a second person closest to the first person; A display method including:

13. On the computer, a distance acquisition function for acquiring a distance between a first person appearing in an image captured by a camera and another person appearing in the image; an image processing function for displaying, on a display device, a display element visually indicating a reference distance in a manner superimposed on the first person in the image; Let them have the image processing function changes a display mode of the display element based on a first distance, which is a distance acquired for a second person closest to the first person; program.

Images