Display control device, display control method, and display control system

Abstract

To enable guidance to an examination location and images relating to the contents of an examination to be displayed in a more appropriate display mode. [Solution] The display control device of this embodiment includes an estimation unit, a recognition processing unit, and a display control unit. The estimation unit estimates the walking state of the subject. The recognition processing unit recognizes objects related to the subject's examination based on an image of the subject. The display control unit displays guidance to the examination location and images related to the examination content on the display device in a display mode based on at least the recognized objects and the walking state.

Description

[Technical Field]

[0001] The present invention relates to a display control device, a display control method, and a display control system. [Background technology]

[0002] Baggage inspections are generally carried out at various locations, such as building entrances, theater entrances, train stations, airports, etc. At airports, etc., digital signage is used to guide people carrying items subject to inspection to the inspection location. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-014522 Summary of the Invention [Problem to be solved by the invention]

[0004] However, if the displayed images guiding the subject to the examination location or the examination contents are out of sync with the subject's examination target or the subject's viewing timing, it may be difficult for the subject to confirm the information in the displayed images.

[0005] The problem to be solved by the present invention is to provide a display control device, a display control method, and a display control system that are capable of displaying images relating to guidance to an examination location and examination contents in a more appropriate display mode. [Means for solving the problem]

[0006] The display control device of this embodiment includes an estimation unit, a recognition processing unit, and a display control unit. The estimation unit estimates the walking state of the subject. The recognition processing unit recognizes objects related to the subject's examination based on an image of the subject. The display control unit displays guidance to the examination location and images related to the examination content on the display device in a display mode based on at least the recognized objects and the walking state. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a block diagram showing the configuration of a display control system according to an embodiment of the present invention. [Figure 2] FIG. 1 is a diagram schematically illustrating an example of the configuration of a display control system. [Figure 3] FIG. 2 is a diagram showing the configuration of the display control system in more detail. [Figure 4] FIG. 1 is a block diagram showing an example of the configuration of a display control device. [Figure 5] FIG. 4 is a diagram showing an example of a human region in an image identified by a region identification unit. [Figure 6] FIG. 10 is a side view showing a marker to be placed in a measurement area. [Figure 7] Schematic diagram showing the time-series positions of a subject walking in the measurement area. [Figure 8] FIG. 10 is a schematic diagram showing another example of marker placement. [Figure 9] FIG. 2 is a diagram schematically illustrating a classifier of a recognition processing unit. [Figure 10] FIG. 10 is a diagram showing an example of an image displayed on the information providing device. [Figure 11] 4 is a flowchart showing an example of the operation of the display control system according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] A display control device, a display control method, and a display control system according to embodiments of the present invention will be described in detail below with reference to the drawings. Note that the embodiments described below are examples of embodiments of the present invention, and the present invention should not be interpreted as being limited to these embodiments. Furthermore, in the drawings referred to in this embodiment, identical parts or parts having similar functions are given the same or similar symbols, and repeated explanations thereof may be omitted. Furthermore, for convenience of explanation, the dimensional ratios of the drawings may differ from the actual ratios, and some components may be omitted from the drawings. (One embodiment)

[0009] 1 is a block diagram showing the configuration of a display control system 1 according to this embodiment. The display control system 1 according to this embodiment is, for example, a system for digital signage.

[0010] An overview of the configuration of a display control system 1 according to one embodiment will be described below with reference to FIGS. 1 to 3. FIG. 1 is a block diagram showing the configuration of the display control system 1 according to this embodiment. The display control system 1 according to this embodiment includes a plurality of imaging devices 10a to 10f, a display control device 20, a plurality of electronic devices 30, a plurality of information providing devices 40, and a data server 50. FIG. 1 also shows a communication network Nw. In the following description, when describing the processing contents of the plurality of imaging devices 10, the plurality of electronic devices 30, and the plurality of information providing devices 40, etc., the following descriptions may be given with references such as 10a to 10f, 30a to 30c, 40a to 40g, etc.

[0011] The imaging device 10 is, for example, a network camera, and generates an image by AD converting an image signal generated by the camera's imaging element. The imaging device 10 according to this embodiment is configured to continuously generate image data in frame units and capture moving image data. The imaging device 10 transmits image data to the display control device 20 and the data server 50 via the communication network Nw, associating the image data with the image capture time and the number of the imaging device 10. The number of the imaging device 10 is associated with, for example, position coordinates. This makes it possible to determine the image capture time and image capture position of the image data. The image data captured by the imaging device 10 is, for example, data composed of a two-dimensional array of R, G, and B pixel values. Note that in this embodiment, the image data used for processing by the display control system 1 may be simply referred to as an image. Furthermore, it is also possible to directly connect each device to each other without using the communication network Nw.

[0012] The display control device 20 recognizes the object of inspection from the items worn or carried by the subject p10a using image data of the subject p10a captured by the imaging device 10. The display control device 20 also presents the recognized object and the walking state of the subject p10a to the information providing device 40 via the electronic device 30 in a display mode based on at least the recognized object. The display mode according to this embodiment includes the content, format, and timing of presentation information such as display images, videos (time-series display images), audio, vibration, and light. The display control device 20 will be described in detail later.

[0013] The electronic device 30 is, for example, a set-top box (STB), and is a control device for controlling the display of the information providing device 40. The electronic device 30 is connected to the information providing device 40, and is a device that presents to the information providing device 40 guidance to the examination location and images, videos, etc. related to the examination contents, based on a display image signal received from the display control device 20 via the communication network Nw. Note that the electronic device 30 may be configured integrally with the information providing device 40.

[0014] The electronic device 30 can also store display content in advance. In this case, it is possible to display pre-stored guidance to the examination location and images, videos, etc. relating to the examination content on the information providing device 40 based on the display image signal received from the display control device 20.

[0015] The information providing device 40 is, for example, an electronic signboard for digital signage. The information providing device 40 displays at least an image among images, sounds, and vibrations related to examination guidance to the subject under the control of the electronic device 30. Note that the information providing device 40 according to this embodiment corresponds to a display device. Furthermore, the information providing device 40 is not limited to an electronic signboard, and may be a display device such as an LCD monitor.

[0016] The data server 50 is, for example, a content management server for digital signage. The display content, order, date, day of the week, and time period for images, videos, etc. provided by the information providing device 40 are set in advance, and instructions are given to the electronic device 30. It is possible to set in advance not only the images to be provided to viewers who have the object of inspection, but also the images to be provided to viewers who do not have the object of inspection, and the images to be provided when there are no viewers within the camera's shooting range.

[0017] FIG. 2 is a diagram schematically illustrating an example configuration of the display control system 1. FIG. 3 is a diagram illustrating the configuration of the display control system 1 in more detail. As illustrated in FIGS. 2 and 3, the imaging device 10a is disposed at a location where it captures an image of a subject P10a walking from a measurement area a100. The display control device 20 uses time-series image data captured by the imaging device 10a to perform a process of estimating the walking state of the subject P10a, including the speed, and a process of recognizing an object of inspection related to the subject P10a. Furthermore, the display control device 20 estimates the time it takes for the subject P10a to reach the viewing area a102 of the information providing device 40a and the viewing area a104 of the information providing device 40b, based on the walking state of the subject P10a, including the speed. Meanwhile, the imaging devices 10b to 10f are disposed at locations where the subject P20 in the waiting areas a200b to a200f of the inspection site can be captured. As a result, the imaging devices 10b to 10f capture images of the subject P20 in the waiting areas a200b to a200f of the inspection site.

[0018] The information providing device 40a presents to the subject P10a, for example, a checkpoint corresponding to the subject P10a's belongings. At this time, the display control device 20 presents information about the checkpoint at a timing based on an estimated time for the subject P10a to arrive at the viewing area a102 of the information providing device 40a. More specifically, the display control device 20 presents to the information providing device 40a an image guiding the subject P10a to a checkpoint with a small number of subjects P20 in the waiting areas a200b to a200f.

[0019] The information providing device 40b presents to the subject P10a, for example, a prior notice corresponding to the test corresponding to the subject P10a's belongings. At this time, the display control device 20 presents information regarding the prior notice at a timing based on the estimated time when the subject P10a will arrive at the viewing area a104 of the information providing device 40b. More specifically, when the display control device 20 recognizes the object of the test related to the subject P10a, it displays an image including information regarding the recognized object on the information providing device 40a. Alternatively, when the display control device 20 does not recognize the object of the test related to the subject P10a, it displays an image including information regarding general guidance to the test location and information regarding the test content on the information providing device 40a. The information providing devices 40c to 40g present information regarding guidance to the test location and information regarding the test content to the subject P20.

[0020] Here, a detailed configuration example of the display control device 20 will be described with reference to Fig. 4. Fig. 4 is a block diagram showing a configuration example of the display control device 20. As shown in Fig. 4, the display control device 20 has a CPU (Central Processing Unit) and is, for example, a computer. The display control device 20 has an acquisition unit 200, an area identification unit 202, an estimation unit 204, a recognition processing unit 206, a display control unit 208, and a storage unit 210. Some or all of these functional units are realized by, for example, a hardware processor such as a CPU executing a program (software) stored in the storage unit 210. For example, the storage unit 210 is configured with a ROM (Read Only Memory), a RAM (Random Access Memory), etc.

[0021] The program may be downloaded from a device (such as an application server) connected via the communication network Nw, or may be stored in a portable storage medium such as an SD card and then installed in the display control device 20. Some or all of the functional units of the display control device 20 may be realized by hardware such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field Programmable Gate Array), or may be realized by a combination of software and hardware.

[0022] The acquiring unit 200 acquires image data from the imaging devices 10a to 10f via the communication network Nw. The acquiring unit 200 may be configured to acquire image data stored in a data server 50 or image data provided via an internet line or the like.

[0023] Fig. 5 is a diagram showing an example of a human region pa10 in an image identified by the region identification unit 202. Fig. 5(a) is an example of an image g10 of the imaging device 10a acquired by the acquisition unit 200. Fig. 5(b) is a diagram showing examples of a human region pa10 and regions da10 to da18 identified by the region identification unit 202 from the image g10.

[0024] 5(b), the region identification unit 202 performs a predetermined calculation process on the image g10 to identify a region including a human region pa10 appearing in the image. The region identification unit 202 according to this embodiment identifies, for example, a rectangular region including a region in the image where a person has been identified as the human region pa10.

[0025] Furthermore, the area classification unit 202 classifies areas da10 to da18, which are to be recognized by a recognition processing unit 206 (described later), based on the human area pa10. That is, these areas da10 to da18 correspond to processing areas for each classifier of the recognition processing unit 206 (described later).

[0026] The region classification unit 202 can classify the human region pa10, regions da10 to da18, etc. using a general method. For example, the region classification unit 202 classifies a difference image in a time-series image and identifies the human region pa10 from the difference image. The region classification unit 202 can also classify the regions da10 to da18, etc. using other methods such as a trained neural network, template matching, a combination of HOG (Histograms of Oriented Gradients) features and SVM (Support Vector Machine), or a background subtraction method.

[0027] Here, an example of processing by the estimation unit 204 will be described with reference to Figs. 6 to 8. Fig. 6 is a schematic side view showing markers M10 to M22 to be placed in the measurement area a100. The markers M10 to M22 are placed in the measurement area a100, for example, at equal intervals, so that they can be captured by the imaging device 10a and recognized as markers.

[0028] 7 is a schematic diagram showing the time-series positions of a subject p10a walking in a measurement area a100. The area identification unit 202 extracts markers from images captured in time series by the imaging device 10a. The estimation unit 204 distinguishes between markers that have been recognized from the captured images and markers M10 to M22 that have become unrecognizable due to being blocked by passing people. As a result, the estimation unit 204 estimates the time at which each marker became unrecognizable and the speed at which the subject moves through each area as the walking state.

[0029] 8 is a schematic diagram showing another example of marker placement. Markers are placed in multiple rows parallel to the aisle, from M10a to M18a and from M10b to M18b. The area identification unit 202 extracts a human area pa10 from images captured in time series by the imaging device 10a. The estimation unit 204 distinguishes between markers that were recognized from the captured images and markers that became unrecognizable due to being blocked by passing people, from M10a to M18a and from M10b to M18b, based on their relationship. The estimation unit 204 estimates the time when each marker became unrecognizable and estimates the speed of movement through each area as the walking state.

[0030] Furthermore, the estimation unit 204 estimates the time it will take for the subject p10a to reach the viewing areas a102 and a104. For example, the estimation unit 204 estimates the time it will take for the subject p10a to reach the viewing areas a102 and a104 based on the estimated speed at which the subject p10a passes through each area. Even if the time it takes to pass between M10 and M22 (see FIG. 7) is the same, if the speed at which the subject p10a passes through each area is increasing, the estimation unit 204 sets a shorter time to reach the viewing areas a102 and a104 (see FIG. 3), and if the speed at which the subject p10a passes through each area is decreasing, the estimation unit 204 sets a longer time to reach the viewing areas a102 and a104. In this way, the estimation unit 204 estimates the time it will take for the subject p10a to reach the viewing areas a102 and a104, reflecting the tendency of the walking speed according to the walking state of the subject p10a.

[0031] The recognition processing unit 206 executes a process of estimating the number of subjects P20 in each waiting area a200b-a200f (see FIG. 3) at the inspection site based on images captured by the multiple image capture devices 10a-10f. In this case, the recognition processing unit 206 estimates the number of subjects P20 in each waiting area a200b-a200f (see FIG. 3) based on, for example, the number of human areas pa10 identified by the area identification unit 202. Furthermore, the multiple image capture devices 10a-10f may be positioned so that they can capture images of the waiting areas a200b-a200f (see FIG. 3) from above. In this case, it is possible to further improve the accuracy of recognition of the human areas pa10.

[0032] Furthermore, the recognition processing unit 206 recognizes an object related to the examination of the subject p10a based on at least the walking state estimated by the estimation unit 204 and the image of the subject p10a. More specifically, examples of the classifiers d10 to d16 of the recognition processing unit 206 will be described using FIG. 9 while referring to FIG. 5.

[0033] 9 is a diagram schematically showing the classifiers d10 to d16 of the recognition processing unit 206. The recognition processing unit 206 has a plurality of classifiers d10 to d16. The classifiers d10 to d16 can be general classifiers, such as a convolutional neural network (CNN).

[0034] The classifier D10 is a shoe classifier that uses an image of a subject P10a to recognize safety shoes, platform shoes, boots, and ankle-covering shoes D10a. For example, sandals D10b are shoes that do not cover the ankles and are not subject to inspection, but are an example of shoes that are likely to be mistaken for safety shoes, platform shoes, boots, and ankle-covering shoes D10a. The classifier D10 performs recognition processing on the foot area DA10.

[0035] The classifier d12 is a classifier for outerwear among clothing, and uses an image of the subject p10a to recognize, for example, a jacket, a coat, a suit, etc. The classifier d12 executes recognition processing on the clothing area da12.

[0036] The classifier d14 is a classifier for liquid containers, and recognizes liquid containers such as plastic bottles using an image of the subject p10a. The classifier d14 performs recognition processing on the liquid container area da14.

[0037] The classifier d16 is a classifier that recognizes what is being held in the hand, and recognizes an umbrella, a stick with a sharp tip such as a bat d16a, a suitcase, etc., using an image of the subject p10a. An umbrella, a stick with a sharp tip such as a bat d16a, a suitcase, etc. are objects that can be checked in. On the other hand, a walking stick d16b is an example that can be carried, but is likely to be mistaken for a stick with a sharp tip d16a. The classifier d16 performs recognition processing on the carrying area da16.

[0038] The classifier d18 (not shown) is a classifier for scarves and hats among clothing, and recognizes items such as scarves and hats worn on the head or neck using an image of the subject p10a. The classifier d18 performs recognition processing on the head area da18.

[0039] The classifiers d10 to d18 generally undergo a learning process using training data. As a result, the classifiers output the result of classifying the category of the object from the input image as a confidence level. The classifiers d10 to d18 use the category with the highest confidence level as the classification result.

[0040] Model data (structure data, trained parameter data, etc.) of the trained classifiers d10 to d18 is stored, for example, in the storage unit 210. The classifiers d10 to d18 each have a feature extraction unit and a classification unit, where the feature extraction unit performs a process of extracting image features from the image of the regions da10 to da18, and the classification unit performs a process of classifying a category indicating the object type based on the image features. The classifiers d10 to d18 output a classification result and a confidence level indicating the likelihood of the classification result for the corresponding category from among multiple categories. For example, the confidence level is assigned in the range of 0.0 to 1.0. The closer the confidence level is to 0.0, the higher the possibility of an incorrect determination, and conversely, the closer it is to 1.0, the higher the possibility of a correct determination.

[0041] When the difference in the confidence between the identified category with the highest confidence among the classifiers d10 to d18 and the identified category with the second highest confidence is less than a predetermined value, the recognition processing unit 206 re-identifies the category by referring to the walking state of the subject p10a estimated by the estimation unit 204.

[0042] For example, if the difference in the confidence between safety shoes, platform shoes, boots, and ankle-covering shoes d10a and sandals d10b is less than a predetermined value, the classifier d10 re-classifies the category by referring to the speed in each region of the estimation unit 204. For example, the maximum speed in each region of the subject p10a wearing sandals d10b is statistically significantly different from the maximum speed in each region of the subject p10a wearing full shoes, platform shoes, boots, and ankle-covering shoes d10a. The recognition processing unit 206 digitizes the maximum speed, for example, as an addition value in the range of 0.0 to 0.2, and adds it to the confidence of sandals d10b. For example, if the maximum speed of the recognition target object is below a predetermined threshold, the addition value is increased within the range of 0.0 to 0.2 as the maximum speed decreases, and added to the confidence of sandals d10b. As a result, if the maximum speed is equal to or less than a predetermined threshold, the confidence value in the recognition result of sandals d10b increases, and the classifier d10 is more likely to identify the object in the underfoot area da10 as sandals d10b. In this way, as a result of re-identification, the recognition processing unit 206 can change the recognized object from safety shoes, platform shoes, boots, or ankle-covering shoes d10a to sandals d10b in accordance with the walking state estimated by the estimation unit 204. In other words, if the person is moving slowly, it is possible to increase the confidence that the object is sandals d10b.

[0043] For example, if the difference in the confidence between the sharp-tipped stick d16a and the walking cane d16b is less than a predetermined value, the classifier d16 re-classifies the category by referring to the speed in each region of the estimation unit 204. For example, there is a statistically significant difference between the maximum speed in each region of the subject p10a carrying the walking cane d16b and the maximum speed in each region of the subject p10a carrying the sharp-tipped stick d16a. The recognition processing unit 206 quantifies the maximum speed, for example, in the range of 0.0 to 0.2, and adds it to the confidence of the walking cane d16b. For example, if the maximum speed of the recognition target object is below a predetermined threshold, the added value is increased within the range of 0.0 to 0.2 as the maximum speed decreases, and added to the confidence of the walking cane d16b. As a result, if the maximum speed is equal to or less than a predetermined threshold, the confidence value in the recognition result of the assisting cane d16b increases, and the classifier d10 is more likely to identify the object in the holding area da16 as the assisting cane d16b. In this way, as a result of re-identification, the recognition processing unit 206 can change the recognized object from the sharp-tipped stick d16a to the assisting cane d16b in accordance with the walking state estimated by the estimation unit 204. In other words, if the person carrying the long object is moving slowly, it is possible to increase the confidence that the object is the cane d16b.

[0044] The display control unit 208 displays, on the information providing device 40b, images related to guidance to the examination location and examination details, in a display format based on the object of the subject p10a recognized by the recognition processing unit 206. At this time, the display control unit 208 displays, on the information providing device 40b, an image related to examination guidance at the time when the subject p10a will arrive at the viewing area a104 estimated by the estimation unit 204 based on the walking state of the subject p10a. Furthermore, the display control unit 208 changes the image related to examination guidance depending on the number of subjects P20 in each of the waiting areas a200b to a200f (see FIG. 3) recognized by the recognition processing unit 206. For example, the display control unit 208 displays, on the information providing device 40b, a mark image that guides the subject P20 to an examination location with fewer subjects P20 in the waiting areas a200b to a200f (see FIG. 3).

[0045] Fig. 10 is a diagram showing example images to be displayed on the information providing device 40b. Fig. 10(a) is an example image when an object requiring advance notification is recognized by the recognition processing unit 206, and Fig. 10(b) is an example image when an object requiring advance notification is not recognized by the recognition processing unit 206. As shown in Fig. 10, the display control unit 208 displays, on the information providing device 40b, an image relating to the examination guidance in a display format based on the object recognized by the recognition processing unit 206. As described above, the object is at least one of clothing, footwear, and belongings.

[0046] More specifically, the display field d100 of the information providing device 40b changes the background color of the text "Please check before security inspection" from red to white, for example, when the recognition processing unit 206 recognizes an object requiring advance notification. For example, when the display field d102 recognizes a jacket, suit, coat, hat, or scarf as clothing, the background color of the text "Please place items in your pockets and your jacket in a tray" changes from white to yellow. Furthermore, the display control unit 208 displays text such as "Go to manned inspection site A / B" as a mark image M100 on the image to guide the subject p10a wearing a jacket, suit, coat, hat, or scarf to the manned inspection site A / B, etc. In this case, the manned inspection site A / B to which the subject p20 is guided can be an inspection site with a small number of subjects p20, as described above. In this way, the display control unit 208 changes the information on the inspection site, such as A / B of the manned inspection site A / B, in the mark image M100 according to the number of subjects P20 in each of the waiting areas a200b to a200f (see FIG. 3), which are multiple areas. That is, the display control unit 208 can display on the information providing device 40b an image in a display mode that guides the subject P20 to an area with fewer people among the waiting areas a200b to a200f.

[0047] Similarly, when a stick with a sharp tip is recognized, the display field d104 changes the background color of the text "Sharp-tipped umbrellas and long items are recommended to be checked in" from white to yellow. Also, the display control unit 208 displays text such as "Go to airline baggage check" as a mark image M102 to guide the target person p10a carrying a stick with a sharp tip to the baggage check.

[0048] Similarly, when a liquid container is recognized, the display field d106 changes the background color of the text "Please remove liquids, such as plastic bottles, from your baggage and place them in a tray" from white to yellow. Furthermore, the display control unit 208 displays text such as "To manned inspection site A / B" as a mark image M104 on the image to guide the subject p10a carrying the liquid container to the manned inspection site A / B, etc. The manned inspection site A / B to which the subject p20 is guided can be an inspection site with a small number of subjects p20, as described above. In this way, the display control unit 208 changes the inspection site information, such as A / B of the manned inspection site A / B, displayed in the mark image M104 according to the number of subjects P20 in each of the waiting areas a200b to a200f (see FIG. 3), which are multiple areas. That is, the display control unit 208 can display, on the information providing device 40b, an image in a display format that guides the subject p10a to an area with a small number of people among the waiting areas a200b to a200f.

[0049] Similarly, when the display field d108 recognizes safety shoes, platform shoes, boots, or ankle-covering shoes, it changes the background color of the text "Safety shoes, platform shoes, boots, ankle-covering shoes, etc. require inspection" from white to yellow. The display control unit 208 also displays text such as "Go to manned inspection site A / B" as a mark image M104 on the image to guide the subject p10a carrying a liquid container to the manned inspection site A / B, etc. The manned inspection site A / B to which the subject p10a is guided can be a site with a small number of subjects p20, as described above. In this way, the display control unit 208 changes the inspection site information, such as A / B of the manned inspection site A / B, in the mark image M106 according to the number of subjects P20 in each of the waiting areas a200b to a200f (see FIG. 3). That is, the display control unit 208 can display on the information providing device 40b an image in a display mode that guides the user to an area with fewer people among the waiting areas a200b to a200f.

[0050] Similarly, the background color of the text in the display field d110, "Security inspections are conducted in accordance with the Aviation Act, so please undergo inspection following the instructions of inspectors and related personnel," changes, for example, from red to white when an object that requires advance notification is recognized by the recognition processing unit 206. In this way, the display control unit 208 can display, on the information providing device 40b, an image related to inspection guidance in a display format based on the object recognized by the recognition processing unit 206.

[0051] Fig. 11 is a flowchart showing an example of the operation of the display control system according to this embodiment. As shown in Fig. 11, first, the imaging devices 10b to 10f capture image data of the subjects P20 in the waiting areas a200b to a200f of the inspection site (step S100). The area identification unit 202 of the display control device 20 identifies the human area pa10 in each of these time-series image data (step S102). Next, the recognition processing unit 206 of the display control device 20 recognizes the number of subjects P20 in the waiting areas a200b to a200f based on the human area pa10 in each of the image data (step S104).

[0052] Next, the imaging device 10a captures time-series image data of the subject P10a passing through the measurement area a100 (step S106). Next, the area identification unit 202 of the display control device 20 identifies the human area pa10 and areas da10 to da16 in each of the time-series image data (step S108).

[0053] Next, the estimation unit 204 of the display control device 20 estimates the walking state of the subject P10a, including the walking speed of each area in the measurement area a100, and the timing of arrival at the viewing areas a102 and a104, based on the positional relationship between the recognized markers and the unrecognized markers in each of the time-series image data (step S110).

[0054] Next, the display control unit 208 of the display control device 20 displays, on the information providing device 40a, an image showing an inspection site according to the number of subjects P20 in the waiting areas a200b to a200f, based on the estimated arrival time to the viewing area a102 (step S112). That is, the display control unit 208 displays, on the information providing device 40a, an image in a display mode that guides the subject P20 to an area with fewer people among the waiting areas a200b to a200f.

[0055] Next, recognition processing unit 206 executes recognition processing for each of areas da10 to da18 in at least one image data among the time-series image data (step S114). When recognition processing unit 206 recognizes an object that requires advance notification (Y in step S116), display control unit 208 of display control device 20 displays an image in an image format corresponding to the recognized object on information providing device 40b based on the estimated arrival time to viewing area a104 (step S118), and ends the processing.

[0056] On the other hand, if the recognition processing unit 206 does not recognize an object that requires advance notification (N in step S116), the display control unit 208 displays an image in an image format that is not based on the recognition object on the information providing device 40b based on the estimated arrival time to the viewing area a104 (step S112), and terminates the processing.

[0057] As described above, according to this embodiment, the recognition processing unit 206 recognizes objects related to the subject P10a's guidance to the examination location and the contents of the examination based on at least the walking state of the subject P10a and images of the subject, and the display control unit 208 displays an image related to the examination guide in a display format based on the recognized objects on the information providing device 40b. This allows the subject P10a to check in advance how prepared he or she is for the examination. This makes it possible to complete the examination at the examination site in a shorter time.

[0058] In addition, the display control unit 208 displays images on the information providing devices 40a and 40b that provide guidance to the examination location and relate to the examination contents in accordance with the estimated arrival time to the viewing areas a102 and a104 that the estimation unit 204 estimates based on the walking state of the subject, thereby reducing the likelihood that subject P10a will miss images relating to the examination contents.

[0059] In this way, it is possible to present information on guidance to the testing location and information on the test contents on the information providing devices 40a and 40b in a display format that matches the test object of the subject P10a and the viewing timing of the subject P10a, etc. This makes it easier for the subject to check information on guidance to the testing location and information on the test contents.

[0060] At least a part of the display control system 1 described in the above embodiment may be configured with hardware or software. If configured with software, a program that realizes at least a part of the functions of the display control system 1 may be stored on a recording medium such as a flexible disk or a CD-ROM and read and executed by a computer. The recording medium is not limited to removable recording media such as magnetic disks and optical disks, but may also be fixed recording media such as hard disk drives and memories.

[0061] A program that realizes at least some of the functions of the display control system 1 may be distributed via a communication line (including wireless communication) such as the Internet. Furthermore, the program may be encrypted, modulated, or compressed and distributed via a wired line or wireless line such as the Internet, or stored on a recording medium.

[0062] At least a part of the display control system 1 may be implemented by one or more processing units. The processing unit is, for example, one or more electronic circuits including a control circuit and an arithmetic circuit. The electronic circuit is realized by an analog or digital circuit, etc. For example, a general-purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), an ASIC, an FPGA, or a combination thereof is possible.

[0063] Although several embodiments of the present invention have been described above, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, and are also included in the inventions and their equivalents as defined in the claims. [Explanation of symbols]

[0064] 1: display control system, 10, 10a to 10f: imaging device, 20: display control device, 40, 40a to 40g: information providing device, 202: area identification unit, 204: estimation unit, 206: recognition processing unit, 208: display control unit, d10b: sandals, d16b: cane, p10a: subject

Claims

1. an estimation unit that estimates a walking state of a subject; a recognition processing unit that recognizes an object related to the examination of the subject based on an image of the subject; a display control unit that displays, on a display device, an image relating to guidance to an examination location and examination details in a display mode based on at least the recognized object and the walking state; A display control device comprising:

2. the estimation unit estimates a timing to display the image on the display device based on the walking state; The display control device according to claim 1 , wherein the display control unit displays the image on the display device based on the timing as the display mode.

3. The display control device according to claim 1 , wherein the recognition processing unit calculates a confidence level for the recognition result, and changes the confidence level according to the walking state.

4. The display control device according to claim 1 , wherein the estimation unit estimates a walking speed for each predetermined section as the walking state based on time-series images of the subject walking through a predetermined area.

5. The display control device according to claim 1 or 2, wherein the display control unit changes the image in accordance with the number of people in each area corresponding to a plurality of inspection sites.

6. The display control device according to claim 5 , wherein the recognition processing unit recognizes the number of people in each area corresponding to the plurality of inspection sites.

7. The display control device according to claim 5 , wherein the display control unit displays, on the display device, an image in a display form that guides the user to an area with a smaller number of people among the areas corresponding to the plurality of inspection sites.

8. The display control device according to claim 1 , wherein the walking state is a walking speed of the subject in a predetermined area.

9. The object is at least one of clothing, footwear, and belongings; The display control device according to claim 1 or 2, wherein the display control unit displays, on the display device, the image relating to guidance to an inspection location and advance notice regarding inspection contents according to the clothing, the footwear, and the belongings.

10. The display control device according to claim 9 , wherein the display control unit displays, on the display device, the image that guides the user to an inspection site according to at least one of the clothing, the footwear, and the belongings.

11. The display control device according to claim 1 , wherein the display control unit changes one of a character and a color of the image and a display timing based on the recognized object and the walking state.

12. The display control device according to claim 1 or 2, wherein the display device is an electronic sign for digital signage, and provides information on what action to take based on the recognized object by video, audio, vibration, or light.

13. The display control device according to claim 1 , further comprising an area identification unit that identifies an area in an image of the subject that the recognition processing unit executes recognition processing on.

14. The display control device according to claim 3 , wherein the recognition processing unit calculates the confidence level for a recognition target, and the recognition target has a confidence level that increases as the maximum speed of the target person decreases.

15. The display control device according to claim 14 , wherein, when the subject is holding a long object that is the recognition target, the recognition processing unit increases the certainty of the cane as the maximum speed of the subject decreases.

16. The display control device according to claim 14 , wherein, when the subject is wearing footwear that is the recognition object, the recognition processing unit increases the certainty that the subject is wearing sandals as the maximum speed of the subject decreases.

17. Estimate the walking state of the subject, Recognizing an object related to the examination of the subject based on an image of the subject; A display control method for displaying on a display device an image relating to guidance to an examination location or examination contents in a display mode based on at least the recognized object and the walking state.

18. an imaging device that captures an image of a target person passing through a predetermined area; an information providing device that displays at least an image among images, sounds, and vibrations related to examination guidance to the subject; a display control device that displays at least an image on the information providing device, The display control device includes: an estimation unit that estimates a walking state of the subject; a recognition processing unit that recognizes an object related to the examination of the subject based on an image of the subject; a display control unit that displays, on a display device, an image relating to guidance to an examination location or examination details in a display mode based on at least the recognized object and the walking state; A display control system.

Images