Display control device, display control method, and display control system

Abstract

This allows for the display of images related to guidance to the examination location and the examination content in a more appropriate format. [Solution] The display control device of this embodiment comprises an estimation unit, a recognition processing unit, and a display control unit. The estimation unit estimates the walking state of the subject. The recognition processing unit recognizes the objects related to the subject's inspection based on an image of the subject. The display control unit displays images on the display device that guide the subject to the inspection location and the contents of the inspection, in a display mode based on the recognized objects and at least the recognized objects among the walking state.

Description

Technical Field

[0004] ,

[0006] , , , ,

[0005] , , , , ,

[0001] Embodiments of the present invention relate to a display control device, a display control method, and a display control system.

Background Art

[0002] Baggage inspection is generally carried out at various locations such as the entrance of a building, the entrance of a theater, a railway station, an airport, etc. At an airport or the like, an electronic signboard for digital signage is used to guide a person having an inspected baggage to the inspection location for baggage inspection.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, if the display image regarding the guidance to the inspection location or the inspection content is misaligned with the inspected object of the person or the viewing timing of the person, etc., it may be difficult for the person to confirm the information of the display image.

[0005] <00000​​​​​​​The display control device of this embodiment comprises an estimation unit, a recognition processing unit, and a display control unit. The estimation unit estimates the walking state of the subject. The recognition processing unit recognizes objects related to the subject's examination based on an image of the subject. The display control unit displays images on the display device that guide the subject to the examination location and the content of the examination, in a display mode based on the recognized objects and at least the recognized objects among the walking state. [Brief explanation of the drawing]

[0007] [Figure 1] A block diagram showing the configuration of the display control system according to this embodiment. [Figure 2] A schematic diagram illustrating an example of the configuration of a display control system. [Figure 3] A diagram showing the configuration of the display control system in more detail. [Figure 4] A block diagram showing an example configuration of a display control device. [Figure 5] A diagram showing an example of a human region in an image identified by the region identification unit. [Figure 6] A schematic side view showing the markers to be placed in the measurement area. [Figure 7] A schematic diagram showing the chronological position of subjects walking within the measurement area. [Figure 8] A schematic diagram showing another example of marker placement. [Figure 9] A schematic diagram showing the classifier of the recognition processing unit. [Figure 10] A diagram showing an example of an image to be displayed on the information display device. [Figure 11] A flowchart illustrating an example of the operation of the display control system according to this embodiment. [Modes for carrying out the invention]

[0008] Hereinafter, a display control device, a display control method, and a display control system according to embodiments of the present invention will be described in detail with reference to the drawings. The embodiments described below are merely examples of embodiments of the present invention, and the present invention is not limited to these embodiments. Furthermore, in the drawings referenced in these embodiments, the same or similar reference numerals are used for identical parts or parts having similar functions, and repeated descriptions may be omitted. Also, the dimensional ratios in the drawings may differ from the actual ratios for illustrative purposes, and some components may be omitted from the drawings. (One embodiment)

[0009] Figure 1 is a block diagram showing the configuration of the display control system 1 according to this embodiment. The display control system 1 according to this embodiment is, for example, a system for digital signage.

[0010] The following describes the general configuration of a display control system 1 according to one embodiment, with reference to Figures 1 to 3. Figure 1 is a block diagram showing the configuration of the display control system 1 according to this embodiment. The display control system 1 according to this embodiment includes a plurality of imaging devices 10a to 10f, a display control device 20, a plurality of electronic devices 30, a plurality of information providing devices 40, and a data server 50. A communication network Nw is also shown in Figure 1. In the following description, when describing the processing content of the plurality of imaging devices 10, a plurality of electronic devices 30, and a plurality of information providing devices 40, etc., the designations 10a to 10f, 30a to 30c, 40a to 40g, etc. may be used.

[0011] The imaging device 10 is, for example, a network camera, and generates an image by performing AD conversion on the image signal generated by the camera's image sensor. The imaging device 10 in this embodiment is configured to continuously generate frame-by-frame image data and capture moving image data. The imaging device 10 transmits image data to the display control device 20 and the data server 50 via the communication network Nw, associating it with the capture time and the imaging device 10 number. The imaging device 10 number is associated with, for example, position coordinates. This makes it possible to determine the capture time and capture location of the image data. The image data captured by the imaging device 10 is, for example, data composed of R, G, and B pixel values ​​in a two-dimensional array. In this embodiment, the image data used for processing by the display control system 1 may simply be referred to as an image. Furthermore, it is also possible to directly connect each device to each other without using the communication network Nw.

[0012] The display control device 20 uses image data of the subject p10a captured by the imaging device 10 to recognize the object to be inspected from the items worn or carried by the subject p10a. The display control device 20 then presents the recognized object and the walking state of the subject p10a to the information providing device 40 via the electronic device 30 in a display mode based on at least the recognized object. The display mode according to this embodiment includes the content, format, and timing of the presentation of the presented information, such as display images, video (time-series display images), sound, vibration, and light. Further details of the display control device 20 will be described later.

[0013] The electronic device 30 is, for example, a set-top box (STB) and is a control device for controlling the display of the information providing device 40. This electronic device 30 is connected to the information providing device 40 and, based on the display image signal received from the display control device 20 via the communication network Nw, presents images, videos, etc., related to guidance to the inspection location and inspection content to the information providing device 40. The electronic device 30 may also be configured as an integrated unit with the information providing device 40.

[0014] In addition, the electronic device 30 can also store display content in advance. In this case, based on the display image signal received from the display control device 20, it is also possible to display images, videos, etc. related to the guidance to the inspection location and the inspection content stored in advance on the information providing device 40.

[0015] The information providing device 40 is, for example, an electronic signboard for digital signage. The information providing device 40 displays at least an image among an image, voice, and vibration related to the inspection guidance to the target person according to the control of the electronic device 30. Note that the information providing device 40 according to the present embodiment corresponds to a display device. Also, the information providing device 40 is not limited to an electronic signboard and may be a display device such as a liquid crystal monitor.

[0016] The data server 50 is, for example, a content management server for digital signage. It sets in advance the images, videos, etc. provided by the information providing device 40, including the display content, order, display date, day of the week, and time zone, and instructs the electronic device 30. It is possible to set in advance not only the videos provided to viewers who have the inspection object but also the videos provided to viewers who do not have the inspection object and the videos provided when there are no viewers within the shooting range of the camera.

[0017] FIG. 2 is a diagram schematically showing a configuration example of the display control system 1. FIG. 3 is a diagram showing the configuration of the display control system 1 in more detail. As shown in FIGS. 2 and 3, the imaging device 10a is arranged at a location where it can capture the subject P10a walking from the measurement area a100. The display control device 20 uses the time-series image data captured by the imaging device 10a to execute a process of estimating the walking state including the speed of the subject P10a and a process of recognizing the inspection object related to the subject P10a. Further, the display control device 20 estimates the time to reach the viewing area a102 of the information providing device 40a and the time to reach the viewing area a104 of the information providing device 40b based on the walking state including the speed of the subject P10a. On the other hand, the imaging devices 10b to 10f are arranged at locations where the subjects P20 in the waiting areas a200b to a200f of the inspection site can be imaged. Thereby, the imaging devices 10b to 10f capture images of the subjects P20 in the waiting areas a200b to a200f of the inspection site.

[0018] The information providing device 40a presents, for example, an inspection site corresponding to the belongings to the subject P10a. At this time, the display control device 20 presents information regarding the inspection site at a timing based on the estimated time for the subject P10a to reach the viewing area a102 of the information providing device 40a. More specifically, the display control device 20 presents an image guiding an inspection site with a small number of subjects P20 in the waiting areas a200b to a200f to the information providing device 40a.

[0019] Information device 40b presents subject P10a with a pre-notification corresponding to the inspection, for example, of items belonging to the subject. At this time, display control device 20 presents information regarding the pre-notification at a timing based on the estimated time when subject P10a will reach the viewing area a104 of information device 40b. More specifically, if display control device 20 recognizes an object to be inspected for subject P10a, it displays an image containing information about the recognized object on information device 40a. Alternatively, if display control device 20 does not recognize an object to be inspected for subject P10a, it displays an image on information device 40a containing information about general guidance to the inspection location and information about the inspection content. Information devices 40c to 40g present subject P20 with information about guidance to the inspection location and information about the inspection content.

[0020] Here, a detailed configuration example of the display control device 20 will be explained using Figure 4. Figure 4 is a block diagram showing a configuration example of the display control device 20. As shown in Figure 4, the display control device 20 has a CPU (Central Processing Unit), which is, for example, a computer. This display control device 20 has an acquisition unit 200, a region identification unit 202, an estimation unit 204, a recognition processing unit 206, a display control unit 208, and a storage unit 210. Some or all of these functional units are realized by, for example, a hardware processor such as a CPU executing a program (software) stored in the storage unit 210. For example, the storage unit 210 is composed of ROM (Read Only Memory), RAM (Random Access Memory), etc.

[0021] The program may be downloaded from a device connected via a communication network (e.g., an application server), or it may be stored on a portable storage medium such as an SD card and installed on the display control device 20. Furthermore, some or all of the functional parts of the display control device 20 may be implemented by hardware such as an LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), or FPGA (Field Programmable Gate Array), or by the cooperation of software and hardware.

[0022] The acquisition unit 200 acquires image data from imaging devices 10a to 10f via a communication network Nw. The acquisition unit 200 may also be configured to acquire image data stored in the data server 50 or image data provided via an internet connection or the like.

[0023] Figure 5 shows an example of a human region pa10 in an image identified by the region identification unit 202. Figure 5(a) is an example of image g10 of the imaging device 10a acquired by the acquisition unit 200. Figure 5(b) shows an example of the human region pa10 and regions da10 to da18 identified by the region identification unit 202 from image g10.

[0024] As shown in Figure 5(b), the region identification unit 202 performs a predetermined calculation on the image g10 to identify the region in the image that includes the region of a person pa10. In this embodiment, for example, the region identification unit 202 identifies a rectangular region in the image that includes the region in which a person is identified as the region of a person pa10.

[0025] Furthermore, the region identification unit 202 identifies regions da10 to da18, which are to be recognized by the recognition processing unit 206 (described later), based on the human region pa10. In other words, these regions da10 to da18 correspond to the processing regions of the recognition processing unit 206, which will be described later, according to the classifier.

[0026] The region identification unit 202 can identify human region pa10, regions da10 to da18, etc., using general methods. For example, the region identification unit 202 identifies difference images from time-series images and identifies human region pa10 from these difference images. In addition, the region identification unit 202 can identify regions da10 to da18, etc., using methods such as pre-trained neural networks, template matching, a combination of HOG (Histograms of Oriented Gradients) features and SVM (Support Vector Machine), or background subtraction.

[0027] Here, we will explain an example of processing by the estimation unit 204 using Figures 6 to 8. Figure 6 is a schematic side view showing markers M10 to M22 placed in the measurement area a100. The markers M10 to M22 are placed, for example, at equal intervals in the measurement area a100, enabling imaging from the imaging device 10a and allowing them to be recognized as markers.

[0028] Figure 7 is a schematic diagram showing the time-series position of a subject p10a walking in the measurement area a100. The region identification unit 202 extracts markers from images captured time-series by the imaging device 10a. The estimation unit 204 distinguishes between markers that were recognized from the captured images and markers M10 to M22 that became unrecognizable due to being obstructed by a passing person. Based on this, the estimation unit 204 estimates the time when each marker became unrecognizable and estimates the speed at which the subject moves through each region as the walking state.

[0029] Figure 8 is a schematic diagram showing another example of marker arrangement. Markers are placed in multiple parallel rows along the passageway, from M10a to M18a and from M10b to M18b. The region identification unit 202 extracts the human region pa10 from images taken in chronological order by the imaging device 10a. The estimation unit 204 distinguishes between markers that were recognized in the captured images and markers that became unrecognizable due to being obstructed by passing people, based on the relationship between M10a to M18a and M10b to M18b. The estimation unit 204 estimates the time when each marker became unrecognizable and estimates the speed of movement through each region as the walking state.

[0030] Furthermore, the estimation unit 204 estimates the time it takes for the subject p10a to reach viewing areas a102 and a104. For example, the estimation unit 204 estimates the time to reach viewing areas a102 and a104 based on the estimated speed at which the subject passes through each region. Even if the time taken to pass between M10 and M22 (see Figure 7) is the same, the estimation unit 204 sets a shorter time to reach viewing areas a102 and a104 (see Figure 3) if the speed at which the subject passes through each region is increasing, and a longer time to reach viewing areas a102 and a104 if the speed at which the subject passes through each region is decreasing. In this way, the estimation unit 204 estimates the time to reach viewing areas a102 and a104 by reflecting the trend of walking speed according to the walking state of the subject p10a.

[0031] The recognition processing unit 206 performs a process to estimate the number of subjects P20 in each waiting area a200b to a200f (see Figure 3) of the inspection area, based on images captured by the multiple imaging devices 10a to 10f. In this case, the recognition processing unit 206 estimates the number of subjects P20 in each waiting area a200b to a200f (see Figure 3) based, for example, on the number of human regions pa10 identified by the region identification unit 202. Alternatively, the multiple imaging devices 10a to 10f may be positioned to image the waiting areas a200b to a200f (see Figure 3) from above. In this case, the accuracy of recognizing human regions pa10 can be further improved.

[0032] Furthermore, the recognition processing unit 206 recognizes objects related to the examination of subject p10a based on the walking state estimated by the estimation unit 204 and at least one of the images captured of subject p10a. For more details, an example of the classifiers d10 to d16 of the recognition processing unit 206 will be explained using Figure 9, with reference to Figure 5.

[0033] Figure 9 is a schematic diagram showing the classifiers d10 to d16 of the recognition processing unit 206. The recognition processing unit 206 has multiple classifiers d10 to d16. Classifiers d10 to d16 can be general classifiers, such as a convolutional neural network (CNN).

[0034] The classifier d10 is a shoe classifier that uses an image of the subject p10a to recognize safety shoes, platform shoes, boots, and ankle-covering shoes d10a. For example, sandals d10b are shoes that do not cover the ankles and are not subject to inspection, but they are an example of shoes that are easily misidentified as safety shoes, platform shoes, boots, and ankle-covering shoes d10a. The classifier d10 performs recognition processing on the area da10 under the feet.

[0035] The classifier d12 is a classifier for outer garments, and uses an image of the subject p10a to recognize, for example, a jacket, coat, suit, etc. The classifier d12 performs recognition processing on the clothing region da12.

[0036] The classifier d14 is a classifier for liquid containers and recognizes liquid containers such as PET bottles using an image of the subject p10a. The classifier d14 performs recognition processing on the liquid container region da14.

[0037] The classifier d16 is a classifier that recognizes what is being held in the hand, and uses an image of the subject p10a to recognize objects such as umbrellas, sharp-tipped sticks d16a such as bats, and suitcases. Umbrellas, sharp-tipped sticks d16a such as bats, and suitcases are objects that are recommended to be checked in. On the other hand, a walking stick d16b is an object that can be carried, but it is an example of something that is easily misidentified as a sharp-tipped stick d16a. The classifier d16 performs recognition processing on the possession area da16.

[0038] The classifier d18 (not shown) is a classifier for scarves and hats among clothing items, and uses an image of the subject p10a to recognize scarves, hats, and other items worn on the head or neck. The classifier d18 performs recognition processing on the head region da18.

[0039] Classifiers d10-d18 are generally trained using training data. This allows them to identify the category of an object from an input image, outputting the result as a confidence score. Classifiers d10-d18 select the category with the highest confidence score as the identification result.

[0040] The model data (structural data and trained parameter data, etc.) of the trained classifiers d10 to d18 are stored, for example, in the storage unit 210. The classifiers d10 to d18 have a feature extraction unit and an identification unit. The feature extraction unit extracts image features from images in the region da10 to da18, and the identification unit identifies categories indicating object types based on the image features. The classifiers d10 to d18 output the identification result and a confidence score indicating the likelihood of the identification for the relevant category from among multiple categories. For example, the confidence score is assigned in the range of 0.0 to 1.0. In this embodiment, the closer the confidence score is to 0.0, the higher the probability of a misclassification, and conversely, the closer it is to 1.0, the higher the probability of a correct classification.

[0041] The recognition processing unit 206, when the difference in confidence between the identification category with the highest confidence level among the classifiers d10 to d18 and the identification category with the second highest confidence level is less than a predetermined value, refers to the walking state of the subject p10a estimated by the estimation unit 204 and re-identifies the category.

[0042] For example, if the difference in confidence between safety shoes, platform shoes, boots, and ankle-covering shoes d10a and sandals d10b is less than a predetermined value, the classifier d10 re-identifies the category by referring to the speed in each region of the estimation unit 204. For example, the maximum speed in each region of subject p10a wearing sandals d10b will have a statistically significant difference from the maximum speed in each region of subject p10a wearing all shoes, platform shoes, boots, and ankle-covering shoes d10a. The recognition processing unit 206 quantifies an add value in the range of 0.0 to 0.2 based on the maximum speed and adds it to the confidence of sandals d10b. For example, if the maximum speed of the recognized object is below a predetermined threshold, the add value is increased in the range of 0.0 to 0.2 as the maximum speed decreases and added to the confidence of sandals d10b. As a result, if the maximum speed is below a predetermined threshold, the confidence value in the recognition result of the sandal d10b increases, and the likelihood of the classifier d10 identifying the object in the foot area da10 as a sandal d10b increases. In this way, as a result of re-identification, the recognition processing unit 206 can change the object to be recognized from safety shoes, platform shoes, boots, ankle shoes d10a to sandal d10b, depending on the walking state estimated by the estimation unit 204. In other words, when moving slowly, it is possible to increase the certainty of identifying the sandal d10b.

[0043] For example, if the difference in confidence between the sharp-tipped rod d16a and the walking aid cane d16b is less than a predetermined value, the classifier d16 re-identifies the category by referring to the speed in each region of the estimation unit 204. For example, the maximum speed in each region of subject p10a holding the walking aid cane d16b is statistically significant compared to the maximum speed in each region of subject p10a holding the sharp-tipped rod d16a. The recognition processing unit 206 quantifies the maximum speed, for example, in the range of 0.0 to 0.2, and adds it to the confidence level of the walking aid cane d16b. For example, if the maximum speed of the recognized object is below a predetermined threshold, the added value is increased in the range of 0.0 to 0.2 as the maximum speed decreases and added to the confidence level of the walking aid cane d16b. As a result, if the maximum speed is below a predetermined threshold, the confidence value in the recognition result of the auxiliary cane d16b increases, and the likelihood of the classifier d10 identifying the object in the holding area da16 as the auxiliary cane d16b increases. In this way, as a result of re-identification, the recognition processing unit 206 can change the object to be recognized from a rod with a sharp tip d16a to an auxiliary cane d16b, according to the walking state estimated by the estimation unit 204. In other words, if the person carrying the long object is moving slowly, it is possible to increase the certainty that it is the cane d16b.

[0044] The display control unit 208 displays images on the information providing device 40b that guide the subject p10a to the inspection location and provide information about the inspection content, based on the object recognized by the recognition processing unit 206. At this time, the display control unit 208 displays images related to inspection guidance on the information providing device 40b at the time it takes for the subject p10a to reach the viewing area a104, which is estimated by the estimation unit 204 based on the subject p10a's walking state. The display control unit 208 also changes the images related to inspection guidance according to the number of subject p20s in each waiting area a200b to a200f (see Figure 3) recognized by the recognition processing unit 206. For example, the display control unit 208 displays a mark image on the information providing device 40b that guides the subject p20 to the inspection area with a small number of subject p20s in the waiting areas a200b to a200f (see Figure 3).

[0045] Figure 10 shows an example of an image displayed on the information providing device 40b. Figure 10(a) is an example of an image when the object requiring prior notification is recognized by the recognition processing unit 206, and Figure 10(b) is an example of an image when the object requiring prior notification is not recognized by the recognition processing unit 206. As shown in Figure 10, the display control unit 208 displays an image related to inspection guidance in a display format based on the object recognized by the recognition processing unit 206 on the information providing device 40b. The object is, as described above, at least one of clothing, footwear, and personal belongings.

[0046] More specifically, in the information provision device 40b, the background color of the text "Please check before security inspection" in the display field d100 changes from red to white, for example, when the recognition processing unit 206 recognizes an object that requires prior notification. For example, in the display field d102, if a jacket, suit, coat, hat, or scarf is recognized as clothing, the background color of the text "Please place items in your pockets and outerwear in the tray" changes from white to yellow. The display control unit 208 also displays text such as "To manned inspection area A / B" as a mark image M100 to guide the person p10a wearing a jacket, suit, coat, hat, or scarf to manned inspection area A / B. In this case, the manned inspection area A / B to which the person is guided can be an inspection area with a small number of persons p20, as described above. In this way, the display control unit 208 changes the information of the inspection area, such as A / B in the manned inspection area A / B, in the mark image M100 according to the number of subjects P20 in each of the multiple waiting areas a200b to a200f (see Figure 3). In other words, the display control unit 208 can display an image on the information providing device 40b that guides the subject to the area with the fewest people among the waiting areas a200b to a200f.

[0047] Similarly, if the display panel d104 detects a rod with a sharp tip, it changes the background color of the text "We recommend checking in long umbrellas with sharp tips" from white to yellow. The display control unit 208 also displays text such as "Go to airline baggage check-in" as a mark image M102 to guide the person p10a carrying a rod with a sharp tip to baggage check-in.

[0048] Similarly, when a liquid container is detected in the display area d106, the background color of the text "Please remove liquids such as PET bottles from your luggage and place them in the tray" changes from white to yellow. The display control unit 208 also displays text such as "To manned inspection area A / B" as a mark image M104 to guide the person p10a carrying a liquid container to manned inspection area A / B. In this case, the manned inspection area A / B to which the person is guided can be an inspection area with a small number of persons p20, as described above. In this way, the display control unit 208 changes the information of the inspection area, such as A / B in the manned inspection area A / B, in the mark image M104 according to the number of persons P20 in each of the multiple waiting areas a200b~a200f (see Figure 3). In other words, the display control unit 208 can display an image in a display format that guides the person to an area with a small number of people among the waiting areas a200b~a200f on the information providing device 40b.

[0049] Similarly, if the display area d108 recognizes safety shoes, platform shoes, boots, or shoes that cover the ankles, it changes the background color of the text "Safety shoes, platform shoes, boots, shoes that cover the ankles, etc. require inspection" from white to yellow. The display control unit 208 also displays text such as "To manned inspection area A / B" as a mark image M104 on the image to guide the person p10a carrying a liquid container to manned inspection area A / B or the like. The manned inspection area A / B to which the person is guided can be an inspection area with a small number of persons p20, as described above. In this way, the display control unit 208 changes the information of the inspection area, such as A / B in the manned inspection area A / B, in the mark image M106 according to the number of persons P20 in each of the multiple waiting areas a200b~a200f (see Figure 3). In other words, the display control unit 208 can display an image on the information providing device 40b that guides people to an area with a small number of people among the waiting areas a200b to a200f.

[0050] Similarly, the background color of the text in display field d110, "Security inspections are conducted in accordance with the Aviation Act, so please follow the instructions of the inspectors and relevant personnel during the inspection," changes from red to white, for example, when the recognition processing unit 206 recognizes an object that requires prior notification. In this way, the display control unit 208 can display an image related to inspection guidance in a display format based on the object recognized by the recognition processing unit 206 to the information providing device 40b.

[0051] Figure 11 is a flowchart illustrating an example of the operation of the display control system according to this embodiment. As shown in Figure 11, first, the imaging devices 10b to 10f capture image data of the subjects P20 in the waiting areas a200b to a200f of the inspection area (step S100). The area identification unit 202 of the display control device 20 identifies the person area pa10 of each of these time-series image data (step S102). Subsequently, the recognition processing unit 206 of the display control device 20 recognizes the number of subjects P20 in the waiting areas a200b to a200f based on the person area pa10 of each image data (step S104).

[0052] Next, the imaging device 10a captures time-series image data of the subject P10a passing through the measurement area a100 (step S106). Then, the region identification unit 202 of the display control device 20 identifies the person region pa10 and regions da10~da16 for each of these time-series image data (step S108).

[0053] Next, the estimation unit 204 of the display control device 20 estimates the walking state of subject P10a, including the walking speed in each area of ​​the measurement area a100, and the timing of arrival at viewing areas a102 and a104, based on the positional relationship between the recognized markers and the unrecognized markers in each time-series image data (step S110).

[0054] Next, the display control unit 208 of the display control device 20 displays an image on the information providing device 40a indicating the inspection area corresponding to the number of subjects P20 in the waiting areas a200b to a200f, based on the estimated arrival time to the viewing area a102 (step S112). That is, the display control unit 208 displays an image on the information providing device 40a in a display format that guides the subject to an area with a small number of people among the waiting areas a200b to a200f.

[0055] Next, the recognition processing unit 206 performs recognition processing for each region da10 to da18 in at least one image data from the time-series image data (step S114). When the recognition processing unit 206 recognizes an object that requires prior notification (Y in step S116), the display control unit 208 of the display control device 20 displays an image in an image format corresponding to the recognized object on the information providing device 40b based on the estimated arrival time to the viewing area a104 (step S118), and terminates the process.

[0056] On the other hand, if the recognition processing unit 206 does not recognize an object that requires prior notification (N in step S116), the display control unit 208 displays an image in an image format that is not based on the recognition object on the information providing device 40b based on the estimated arrival time to the viewing area a104 (step S112), and terminates the process.

[0057] As described above, according to this embodiment, the recognition processing unit 206 recognizes the objects related to guiding the subject P10a to the inspection location and the content of the inspection, based on the walking state of the subject P10a and at least one of the images captured of the subject. The display control unit 208 displays an image of the inspection guidance in a display format based on the recognized objects on the information providing device 40b. This makes it possible for the subject P10a to check the preparation of their inspection content in advance. As a result, it becomes possible to complete the inspection at the inspection site in a shorter time.

[0058] Furthermore, the display control unit 208 displays images related to guidance to the examination location and the examination content on the information provision devices 40a and 40b according to the estimated arrival time to viewing areas a102 and a104 estimated by the estimation unit 204 based on the subject's walking state, thereby reducing the likelihood of subject P10a overlooking images related to their own examination.

[0059] In this way, information regarding guidance to the inspection location and the content of the inspection can be presented to the information provision devices 40a and 40b in a display manner that matches the object being inspected by subject P10a and the timing of subject P10a's viewing. This makes it easier for the subject to check information regarding guidance to the inspection location and the content of the inspection.

[0060] At least a part of the display control system 1 described in the above-described embodiment may be configured as hardware or as software. If configured as software, a program that implements at least some of the functions of the display control system 1 may be stored on a recording medium such as a flexible disk or CD-ROM, loaded into a computer, and executed. The recording medium is not limited to removable ones such as magnetic disks or optical disks, but may also be a fixed recording medium such as a hard disk drive or memory.

[0061] Furthermore, a program that implements at least some of the functions of the display control system 1 may be distributed via communication lines such as the Internet (including wireless communication). In addition, the program may be encrypted, modulated, or compressed and distributed via wired or wireless lines such as the Internet, or stored on a recording medium.

[0062] Furthermore, at least a part of the display control system 1 may be implemented by one or more processing units. The processing unit is, for example, one or more electronic circuits including a control circuit and an arithmetic circuit. The electronic circuit can be implemented as an analog or digital circuit, etc. For example, a general-purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), an ASIC, an FPGA, or a combination thereof is possible.

[0063] Although several embodiments of the present invention have been described above, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. [Explanation of Symbols]

[0064] 1: Display control system, 10, 10a~10f: Imaging device, 20: Display control device, 40, 40a~40g: Information providing device, 202: Area identification unit, 204: Estimation unit, 206: Recognition processing unit, 208: Display control unit, d10b: Sandal, d16b: Cane, p10a: Subject

Claims

1. An estimation unit that estimates the walking state of the subject, A recognition processing unit recognizes objects related to the inspection of the subject based on an image of the subject, A display control unit that displays images on a display device that guides the user to the inspection location and the content of the inspection, in a display manner based on at least the recognized object and the walking state, A display control device equipped with the following:

2. The estimation unit estimates the timing for displaying the image on the display device based on the walking state. The display control device according to claim 1, wherein the display control unit displays the image on the display device based on the timing as the display mode.

3. The display control device according to claim 1 or 2, wherein the recognition processing unit calculates a degree of confidence for the recognition result and changes the degree of confidence according to the walking state.

4. The display control device according to claim 1 or 2, wherein the estimation unit estimates the walking speed for each predetermined section as the walking state based on time-series images of the subject passing through a predetermined area.

5. The display control device according to claim 1 or 2, wherein the display control unit changes the image according to the number of people in each area corresponding to a plurality of inspection areas.

6. The display control device according to claim 5, wherein the recognition processing unit recognizes the number of people in each area corresponding to the plurality of inspection areas.

7. The display control device according to claim 5 or 6, wherein the display control unit displays an image on the display device that guides people to the area with the fewest number of people among the areas corresponding to the plurality of inspection areas.

8. The display control device according to claim 1 or 2, wherein the walking state is the walking speed of the subject in a predetermined area.

9. The aforementioned object is at least one of the following: clothing, footwear, and personal belongings. The display control device according to claim 1 or 2, wherein the display control unit displays images on the display device relating to guidance to an inspection location and prior notification regarding the content of the inspection, according to the clothing, footwear, and belongings.

10. The display control device according to claim 9, wherein the display control unit displays the image on the display device that guides the user to an inspection area corresponding to at least one of the clothing, footwear, and belongings.

11. The display control device according to claim 1 or 2, wherein the display control unit changes the display characters, colors, and display timing of the image based on the recognized object and the walking state.

12. The display device is an electronic sign for digital signage, and provides information on actions to be taken based on the recognized object using video, sound, vibration, and light, as described in claim 1 or 2.

13. The display control device according to claim 1 or 2, further comprising a region identification unit that identifies a region from an image of the subject that the recognition processing unit performs recognition processing on.

14. The display control device according to claim 3, wherein the recognition processing unit calculates the degree of confidence for the recognition target, and the degree of confidence for the recognition target increases as the maximum speed of the target decreases.

15. The display control device according to claim 14, wherein the recognition processing unit increases the confidence level of the cane as the maximum speed of the subject decreases when the subject is holding a long object that is the object to be recognized.

16. The display control device according to claim 14, wherein the recognition processing unit increases the confidence level of the sandals as the maximum speed of the subject decreases when the subject is wearing footwear that is the object to be recognized.

17. To estimate the walking condition of the subject, Based on the image of the subject, the object related to the subject's inspection is recognized. A display control method for displaying images on a display device that guide the user to an inspection location or relate to the inspection content, in a display mode based on at least the recognized object among the recognized object and the walking state.

18. An imaging device that images a subject passing through a designated area, An information providing device that displays at least an image among images, sounds, and vibrations related to the inspection guidance to the aforementioned subject, A display control system comprising a display control device that displays at least an image on the information providing device, The aforementioned display control device is An estimation unit for estimating the walking state of the subject, A recognition processing unit recognizes objects related to the inspection of the subject based on an image of the subject, A display control unit that displays images on a display device that guides the user to the inspection location and the content of the inspection, in a display mode based on at least the recognized object among the recognized object and walking state, A display control system having the following features.

Images