Guidance device for visually impaired people and shopping support system for visually impaired people

Abstract

A relatively low-cost guide device for visually impaired persons and a shopping support system for visually impaired persons are provided. [Solution] An easy-to-use guidance device for visually impaired persons, etc., comprising a processor, a processing unit connected to the processor and having a main memory in which at least characteristic parts of the appearance of a specified object are pre-stored, linked to the object's ID, a headset worn on the user's head, and a wearable device worn on or near the wrist of one of the user's arms. The processor receives an image signal from a camera which is an object imaging unit provided in the wearable device, and confirms whether the object is present within the field of view of the object imaging unit. It also creates presence / absence and distance signals based on distance signals from a distance sensor also provided in the wearable device, and notifies the user of the contents of these signals through a speaker such as an earphone provided in the headset, thereby providing safe walking navigation.

Description

[Technical Field]

[0001] The present invention relates to a guidance device for visually impaired persons and a shopping support system for visually impaired persons, and more specifically to a guidance device for visually impaired persons that enables visually impaired persons to walk safely on roads, for example, and a shopping support system for visually impaired persons that enables visually impaired persons to actually pick up products when shopping at supermarkets, convenience stores, etc. [Background technology]

[0002] As a guidance device for the visually impaired and the like, particularly a shopping support system for the visually impaired and the like, an infrared guidance system and a user guidance terminal device have been proposed in Japanese Patent Application Laid-Open No. 2006-258591.

[0003] The infrared guidance system proposed in this publication is characterized by comprising one or both of a left-side transmitter that emits infrared rays carrying individual information from the left side of the user's front direction in a direction intersecting the front direction and a right-side transmitter that emits infrared rays carrying individual information from the right side of the user's front direction in a direction intersecting the front direction; an infrared receiving device that has a front infrared receiving unit that receives infrared rays from the front, a left infrared receiving unit that receives infrared rays from the left side, and a right infrared receiving unit that receives infrared rays from the right side, and that has the shape of an item that is worn above the user's neck or a shape that can be attached to such an item; a signal processing unit that creates a guidance signal based on the individual information carried on the infrared rays received by the front infrared receiving unit, the left infrared receiving unit, or the right infrared receiving unit; and an audio output device that announces guidance information by voice based on the guidance signal.

[0004] The user guidance terminal device is characterized by comprising an infrared receiving device that has a front infrared receiving unit that receives infrared rays from the front, a left infrared receiving unit that receives infrared rays from the left side, and a right infrared receiving unit that receives infrared rays from the right side, and that has the shape of an item that is worn above the user's neck or a shape that can be attached to said item, a signal processing device that creates a guidance signal based on the infrared rays received by the front infrared receiving unit, the left infrared receiving unit, or the right infrared receiving unit, and an audio output device that announces guidance information by voice based on the guidance signal.

[0005] The infrared guidance system and user guidance terminal device configured as described above have the advantage that visually impaired people can obtain information about stores on the left and right sides by using the information transmitted by infrared rays while facing forward. By utilizing this technology, it is possible to construct a device or system that guides visually impaired shoppers to the shelves where the desired products are displayed when shopping at convenience stores (hereinafter referred to as "convenience stores").

[0006] However, the infrared guidance system and user guidance terminal device described above only go so far, making it difficult for visually impaired shoppers to pick up the desired product and shop casually on their own without the help of an assistant. Furthermore, an infrared emitting device must be installed for each target object, which poses a problem of considerable installation work and cost.

[0007] As for a device for the visually impaired, there is one proposed in European Patent Publication No. 002490155. However, this proposed device is not intended for the totally blind, but is merely a visual support system intended for visually impaired people who have some degree of vision. Its target audience is different from that of the device of the present invention, and therefore its structure and configuration are also significantly different. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] Japanese Patent Application Laid-Open No. 2006-258591 [Patent Document 2] European Patent Publication No. 002490155 Summary of the Invention [Problem to be solved by the invention]

[0009] The present invention aims to provide a relatively low-cost guidance device and shopping support system for the visually impaired, which solves the problems with the above-mentioned infrared guidance system and user guidance terminal device and enables visually impaired shoppers to easily pick up the products they want and shop by themselves without the help of an assistant. [Means for solving the problem]

[0010] The above problems are solved by the following guidance devices for the visually impaired and shopping support systems for the visually impaired described in (1) to (11). (1) A guidance device for visually impaired people that can be easily used by users who are visually impaired, etc. a processing unit connected to the processor and having a main memory in which at least a feature portion of the appearance of a predetermined object is stored in advance in association with an ID of the object; a headset worn on the user's head; A wearable device is provided which is attached to the wrist of one arm of a user or in the vicinity thereof, The headset comprises: an object indicating means for outputting a name signal representing the name of the object input by a user to the processing unit; an information transmission means for transmitting information to a user at least by voice; a surrounding environment imaging unit that continuously captures images of the user's surroundings and transmits surrounding environment image signals to the processing unit; The wearable device is an object imaging unit for continuously imaging an object and outputting an object image signal; a distance sensor that measures the distance between the object imaged by the object image capturing unit and a specific part of the user and outputs a distance signal; The processing unit an ID memory that stores an ID for each of the objects; an ID determination unit that receives a name signal from the object specifying means and compares the name signal with the ID memory to read and determine the ID of the object; a temporary storage memory for checking the determined ID against the main memory, reading the characteristic part of the object from the main memory, and temporarily storing the characteristic part; an object detection unit that detects the presence and position of an object in the surrounding environment by comparing the surrounding environment image signal from the surrounding environment imaging unit with the characteristic parts stored in the temporary storage memory; an output audio signal creation unit connected to the object detection unit, which creates audio data representing audio related to the detection of the object in response to an instruction from the object detection unit, and outputs the audio data to the information transmission means of the headset; The processor further comprises: the object detection unit has a function of specifying the direction and approximate distance of the object from the user based on the position, size, etc. of the object in the image when the object is detected, while the output audio signal creation unit receives the direction and approximate distance of the object from the object detection unit, creates a primary detection signal indicating these, and outputs this to the information transmission means; The processor receives an image signal from the object imaging unit, confirms whether an object is present within the field of view of the object imaging unit, and generates presence / absence and distance signals based on the distance signal from the distance sensor, and transmits them to the output audio signal creation unit. The output audio signal creation unit creates presence / absence and distance audio signals based on the received presence / absence and distance signals, and outputs them to the information transmission means. A guidance device for visually impaired people, etc. (2) The guidance device for visually impaired persons, etc., according to (1), wherein the surrounding environment imaging unit is a wide-angle video camera, and the object imaging unit is an ordinary video camera or a narrow-angle camera. (3) The guidance device for visually impaired persons of (1) above, in which the surrounding environment imaging unit is incorporated not in the headset but in a pouch or the like worn by the user. (4) A guidance device for visually impaired persons, etc., according to (1) above, wherein the object indicating means is a microphone that gives instructions by voice. (5) A shopping support system for visually impaired persons, etc., comprising the guidance device for visually impaired persons, etc. of (1), when a shopper who is a user of the device, such as a visually impaired person, purchases products arranged on shelves at a store such as a convenience store, The ID memory stores in advance product IDs that identify products handled at the store, The processing unit is located in a store or in a shopping cart used by a shopper; the main memory stores at least a characteristic part of the external appearance of the product itself or the external appearance of the product package handled by the store, in association with the product ID of the product; the object specifying means is a microphone, and outputs a product name signal representing the name of the target product input by the shopper to the processing unit; the ID determination unit receives the product name signal from the object indicating means, checks the product name signal against the ID memory, and determines the product ID of the product represented by the product name signal; the temporary storage memory checks the product ID determined by the ID determination unit against the main memory, and temporarily stores at least a characteristic portion of the appearance form associated with the product ID based on the product ID; the surrounding environment imaging unit is provided in the shopping cart, and captures images of the external appearance of the products or the external appearance of the packages placed on the product shelves, and transmits image signals representing the images of the external appearances to the processing unit; the object detection unit compares the transmitted image signal with at least the characteristic portion stored in the temporary storage memory, and when a match is found, notifies the shopper of the recognition of the target product together with the general direction from the shopper; the object direction / distance specifying means specifies the direction and distance of the object relative to the user and outputs an object direction / distance signal; The processing unit receives a positional relationship signal output by the object direction / distance identification means and informs the shopper of the direction and distance to the target product indicated by the positional relationship signal. (6) The shopping support system for visually impaired persons, etc., according to (5) above, wherein the storage unit further stores the price of the product, linked to the product ID. (7) The shopping support system for visually impaired persons, etc., as set forth in (6) above, wherein the processing unit reads out the price of the product from the storage unit for each identified product ID and transmits the price to the store's cash register system. (8) The shopping support system for visually impaired persons, etc., according to (5) above, wherein the main memory further stores a 3D image of the appearance of the product itself or the product's packaging, linked to a product ID. (9) When the target product is not within the field of view of the surrounding environment imaging unit and the target imaging unit, the target detection unit transmits this information to the audio signal creation unit, and the audio signal creation unit generates an audio signal to inform the shopper that the target product is not within the field of view. (5) Shopping support system for visually impaired persons, etc. (10) The shopping support system for visually impaired persons, etc., according to (5) above, wherein the surrounding environment imaging unit is a wide-angle camera such as a 360-degree camera. (11) The shopping support system for visually impaired persons, etc., as described in (5) above, further includes an imaging unit height position adjustment device that adjusts the height position of the imaging unit, so that the height of the target products can be adjusted depending on the height of the shelf on which the target products are arranged, etc. [Effects of the Invention]

[0011] In the guidance device and shopping support system for the visually impaired and the like configured as described above, objects are captured using an imaging means, a camera, which serves as a substitute for the visual sense of the visually impaired and the like, so it becomes possible for the visually impaired and the like to walk around town or to shop in stores by picking up specific products without transmitting information indicating the attributes and contents of each object. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a schematic diagram of a guidance device for visually impaired people according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram for explaining the function of the guidance device for the visually impaired and the like shown in FIG. [Figure 3A] FIG. 3A is a flowchart illustrating the operation of the control unit of the guidance device for the visually impaired and the like according to the embodiment of the present invention. [Figure 3B] FIG. 3B is a flowchart illustrating the operation of the control unit of the guidance device for the visually impaired and the like according to the embodiment of the present invention, and is a continuation of FIG. 3A. [Figure 4] FIG. 4 is a system diagram of a shopping support system for visually impaired people and the like according to an embodiment of the present invention. [Figure 5] FIG. 5 is a block diagram for explaining the functions of the shopping support system for the visually impaired and the like shown in FIG. [Figure 6A] FIG. 6A is a flowchart for explaining the operation of the control unit of the shopping support system for visually impaired persons and the like according to the embodiment of the present invention. [Figure 6B]FIG. 6B is a flowchart illustrating the operation of the control unit of the shopping support system for visually impaired persons and the like according to the embodiment of the present invention, and is a continuation of FIG. 6A. [Figure 7] FIG. 7 is a cash register flowchart for explaining the operation of the control unit of the shopping support system for visually impaired persons and the like according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, a guidance device 10 for the visually impaired and the like according to an embodiment of the present invention will be specifically described with reference to the accompanying drawings. In this specification, the term "visually impaired and the like" broadly refers to a person who is totally blind, an infant who cannot distinguish between objects, and the like, examples of which are described below. Here is an example of visual impairment. · Total blindness: A state in which the person has completely lost their sight. Since visual information cannot be used, Braille and audio guidance are important means of support. Low vision: A condition in which vision is partially impaired. Vision correction devices (glasses or contact lenses), magnifying reading devices, and audio guidance are effective. Visual field constriction: A narrowing of the visual field, which may involve preservation of central or peripheral vision. Examples include tunnel vision (loss of peripheral vision with preservation of central vision) and peripheral vision loss (loss of central vision with preservation of peripheral vision). Night blindness: A condition in which vision is impaired at night or in the dark. Vision may be fine in bright light, but may be almost non-functional in the dark. Others: extremely nearsighted people, extremely farsighted people (presbyopia). In this specification, such visually impaired people may also be referred to as users.

[0014] The guidance device 10 for the visually impaired is a device that can be easily used by a user P who is a visually impaired person, and includes a processing unit 20 that is worn by the user P like a waist pouch, a headset 40 that is worn on the head of the user P, and a wearable device 60 that is worn on or near the wrist of one of the arms of the user P. Note that at least a part of the processing unit 20 may be provided in the headset 40 or the like.

[0015] The processing unit 20 includes a processor 22 for performing various calculations and processes to guide and assist the user P, and a main memory 30 connected to the processor 22, which stores in advance feature data representing at least the characteristic features of the appearance of a predetermined object, linked to the ID of the object. If the object is a push-button traffic light S as shown in FIG. 1, it is equipped with a push-button box PB, and a circular push button pb operated by the user is located at a predetermined position on the surface of the push-button box PB. In this case, the characteristic features of the object are the configuration of the push button pb, such as shape: circular, color: red, position: lower part of the push-button box PB, centered in the left-right direction, etc., and the main memory stores this data linked to the ID of the object.

[0016] The headset 40 includes a microphone 42 as an object specifying means for outputting a name signal representing the name of an object input by a user to the processing unit 20; The device is equipped with a speaker 44 such as an earphone as an information transmission means for transmitting information to the user at least by voice, and a first camera 46 such as a 360-degree camera or wide-angle camera as a surrounding environment imaging unit that continuously captures images of the user's surrounding environment and transmits surrounding environment image signals to the processing unit.

[0017] The wearable device 60 is equipped with a second camera 62, which is an object imaging unit that continuously images an object at a closer distance than the first camera 46 and outputs an object close-range image signal, and a distance sensor 64 that measures the distance between the object imaged by the second camera 62 and a specific part of the user, for example, the second camera 62 or the hand at the end of the wrist of the user P wearing the wearable device 60, and outputs a distance signal. The second camera 62 is preferably one with a normal field of view or a narrow-angle camera.

[0018] The processing unit 20 further includes an input / output interface 32 for inputting and outputting signals between the processor 22 and the headset 40 or the wearable device 60, and an ID memory 34 connected to the processor 22 and storing an ID for each object.

[0019] The processor 22 comprises an ID determination unit 23 that receives a name signal from the microphone 42 and compares it with the ID memory 34 to read and determine the ID of the object, a reading unit 24 that compares the determined ID with the main memory and reads out the characteristic part data of the object from this main memory, and a temporary storage memory 25 that temporarily stores the characteristic part data read out by the reading unit 24.

[0020] The processor 22 further includes an object detection unit 26 that compares the surrounding environment image signal from the first camera 46 with the feature data stored in the temporary storage memory 25 to detect the presence of an object and its position in the surrounding environment, and an output audio signal creation unit 27 that is connected to the object detection unit 26 and creates audio data indicating audio related to the detection of the object in response to instructions from the object detection unit 26 and outputs the audio data to the speaker 44 of the headset 40.

[0021] The object detection unit 26 preferably has a function for, upon detecting an object, determining the direction and approximate distance of the object from the user P based on the object's position, size, etc. in the image. A distance sensor may be provided separately to determine the distance. The output audio signal creation unit 27 creates a primary detection signal indicating the direction and approximate distance of the object from the object detection unit 26 and outputs the signal to the speaker 44 of the headset 40. The guidance device 10 for visually impaired persons, etc., includes a switching unit (not shown) that switches the output means for the processing unit 20 from the headset 40 to the wearable device 60 when the user P determines that the object's distance is within a predetermined range after learning the direction and distance of the object from the audio output from the speaker 44 based on the primary detection signal. This switching unit may be capable of automatically switching when the distance indicated by the primary detection signal falls within the predetermined range.

[0022] The processor further includes an object presence confirmation unit 28 that receives an image signal from a second camera 62, which is an object imaging unit, confirms whether an object is present within the field of view of the second camera 62, determines the distance based on the distance signal from the distance sensor 64, and generates presence / absence and distance signals. The object presence confirmation unit 28 transmits the generated signals to the output audio signal generation unit, which generates presence / absence and distance audio signals based on the received presence / absence and distance signals and outputs them to the speaker 44.

[0023] Next, the operation of the processing unit 20 and the processor 22 will be described with reference to the flowcharts of Figures 3A and 3B. Note that the explanation of these flowcharts is mainly from the perspective of the processor 22.

[0024] First, as shown in the flowchart of Figure 3A, a name signal of an object (here, the object is assumed to be a pedestrian crossing traffic signal that accepts a pedestrian's request to cross by pressing a button on a push button box) is received from microphone 42 of headset 40 (step 1), and this is checked against ID memory 34 to determine whether there is an ID that matches the name signal (step 2). If the determination is NO, a prompt signal is output to prompt user P to enter a new object signal (step 3).

[0025] On the other hand, if the determination is YES, an ID is determined and the ID of the object is read from the ID memory 34 (step 4). Next, the read ID is checked against the main memory 30, and the characteristic part data of the object linked to the ID (shape characteristics of the push button box, shape characteristics of the push button (circular), positional relationship on the push button box, etc.) is read from the main memory 30 (step 5). The read characteristic part data is then temporarily stored (memorized) in the temporary storage memory 25 (step 6).

[0026] 3B, a surrounding environment image signal is input from the first camera 46 (step 7), and then this surrounding environment image signal is compared with the feature data stored in the temporary storage memory 25 to determine whether the feature data exists in the surrounding environment image signal (step 8). If the presence is confirmed (i.e., if the determination is YES), the presence of the object is recognized (step 9), and if the determination is NO, the process returns to step 7. This loop is repeated until the object is recognized.

[0027] When the presence of an object is recognized (step 9), in step 10, the output audio signal creation unit 27 creates an audio signal to notify this recognition, and then, when the object detection unit 26 detects an object, it determines the direction and approximate distance of the object from the user P based on the position, size, etc. of the object in the image, and creates and outputs a direction / distance signal, i.e., a primary detection signal (step 11).

[0028] Based on the above, the user P gradually approaches the object, and from the primary detection signal, determines the distance to the object and judges whether the distance to the object is within a predetermined range (step 12). If the judgment is NO, the process returns to step 9. If the judgment is YES, the camera etc. is automatically switched to that of the wearable device (step 13). This switching may be configured in any way.

[0029] The processor receives an image signal from the second camera 62, which is an object imaging unit (step 14), and while confirming that the object is present within the field of view of the second camera 62, determines the distance based on the distance signal from the distance sensor 64, creates a presence / absence and distance signal or a distance signal, and outputs the audio signal to the user P's speaker 44 (step 15).

[0030] Based on the above, user P gradually approaches the object (repeats steps 14 and 15 until the judgment in step 16 becomes YES), and when user P's hand (fingers) touches the object, i.e., presses the traffic light push button, the judgment in step 16 becomes YES and the control ends. The user can now press the target traffic light button without seeing the button.

[0031] Next, a shopping support system 100 for the visually impaired and others according to an embodiment of the present invention will be described in detail with reference to Figure 4 and subsequent drawings. In the description of the shopping support system 100 for the visually impaired and others, members and parts having the same or similar functions as those of the guidance device 10 for the visually impaired and others will be denoted by the same reference numerals. In the shopping support system 100 for the visually impaired, the processing unit 20 is configured as a PC or the like installed in the store itself or in a shopping cart C installed in the store. The processing unit 20 may have a waist pouch-like structure, as in the case of the guidance device for the visually impaired. The headset 40 and wearable device 60 are worn on the head of the visually impaired shopper, or on the wrist of the hand that actually holds the product.

[0032] The processing unit 20 has a processor 22 for performing various calculations and processing to guide and assist the user (shopper) P in shopping, and a main memory 30 connected to this processor 22, which stores in advance characteristic data representing at least the characteristic parts of the appearance of the target product G or its packaging (bag, box, etc.) and the attributes of the target product, linked to the ID of the target product.

[0033] Here, the attributes of the target product include the height of the shelf on which the product is placed, the price, and campaign information for the product (discount sale, double points), etc.

[0034] Similar to the guidance device 10 for the visually impaired, the headset 40 includes a microphone 42 as a target product indicating means for outputting a name signal representing the name of the target product (e.g., anpan) input by the shopper P to the processing unit 20, and a speaker 44 such as an earphone as an information transmitting means for transmitting information to the user at least by voice. The headset 40 may also include the first camera in the guidance device 10 for the visually impaired.

[0035] The shopping cart C is equipped with a first camera 46, such as a 360-degree camera or a wide-angle camera, which serves as a surrounding environment imaging unit that continuously captures images of the user's surroundings and transmits surrounding environment image signals to the processing unit. This first camera 46 is preferably attached to a support column of the shopping cart C so that it can move up and down, and is automatically adjusted to the height of the shelf on which the desired product is placed, which is stored as an attribute of the desired product. This first camera 46 allows a visually impaired shopper to locate a desired product while walking along the aisle between product display shelves.

[0036] The wearable device 60 is equipped with a second camera 62, which is a target product imaging unit that continuously images the target product at a closer distance than the first camera 46 and outputs a target close-distance image signal, and a distance sensor 64 that measures the distance between the target product imaged by the second camera 62 and a specific part of the user, for example, the second camera 62 or the hand at the end of the wrist of the user P wearing the wearable device 60, and outputs a distance signal. The second camera 62 is preferably one with a normal field of view or a narrow-angle camera.

[0037] The processing unit 20 further includes an input / output interface 32 for inputting and outputting signals between the processor 22 and the headset 40 or the wearable device 60, and an ID memory 34 connected to the processor 22 and storing the ID of each target product.

[0038] The processor 22 includes an ID determination unit 23 that receives a name signal of the target product from the microphone 42 and compares it with the ID memory 34 to read and determine the ID of the target product, a reading unit 24 that compares the determined ID with the main memory and reads out the characteristic part data and attribute data of the target product from this main memory, and a temporary storage memory 25 that temporarily stores the characteristic part data and attribute data read by the reading unit 24.

[0039] The processor 22 further includes a target product detection unit 26 that compares the surrounding environment image signal from the first camera 46 with the feature data stored in the temporary storage memory 25 to detect the presence of a target product and its position in the surrounding environment, and an output audio signal creation unit 27 that is connected to the target product detection unit 26 and creates audio data indicating audio related to the detection of the target product in response to an instruction from the target product detection unit 26, and outputs the audio data to the speaker 44 of the headset 40. At the same time, data indicating that the product is a discounted product, etc., from the temporary storage memory 25 may be read out, and information indicated by this data may be included in the audio data.

[0040] The target product detection unit 26 preferably has a function of, when detecting a target product, determining the direction and approximate distance of the target product from the user P based on the position, size, etc. of the target product in the image. The output audio signal creation unit 27 creates a primary detection signal indicating the direction and approximate distance of the target product from the target product detection unit 26, and outputs this to the speaker 44 of the headset 40. The shopping guide device 100 for visually impaired persons, etc., includes switching means (not shown) that switches the means for outputting to the processing unit 20 from the shopping cart C to the wearable device 60 when the user P learns the direction and distance of the target product from the sound emitted from the speaker 44 by the primary detection signal and determines that the distance to the target product is within a predetermined range. This switching means may be capable of automatically performing the switching when the distance indicated by the primary detection signal falls within the predetermined range.

[0041] As a result of the switching, the processor now receives an image signal from the second camera 62, which is a target product imaging unit, and confirms whether the target product is present within the field of view of the second camera 62. It also specifies the distance based on the distance signal from the distance sensor 64 and generates presence / absence and distance signals. The target product presence confirmation unit 28 transmits the generated signal to the output audio signal generation unit, and the output audio signal generation unit generates presence / absence and distance audio signals based on the received presence / absence and distance signals and outputs them to the speaker 44.

[0042] Next, the operation of the processing unit 20 and the processor 22 will be described with reference to the flowcharts of Figures 6A and 6B. Note that the description of these flowcharts is primarily from the perspective of the processor 22.

[0043] As shown in the flowchart of Figure 6A, first, a name signal of the target product (here, the target product is assumed to be Tsubu Anpan) is received from microphone 42 of headset 40 (step 1), and this is compared with ID memory 34 to search and determine whether there is an ID that matches the name signal (step 2). If the determination is NO, a prompt signal is output to prompt user P to specify a new target product, and this is repeated until the determination is YES (step 3). When the store carries two types of anpan, "Tsubu Anpan" and "Smooth Anpan," if the request is simply for "Anpan," the shopper is notified of the above and prompted to specify again.

[0044] On the other hand, if the determination is YES, an ID is determined and the ID of the target product, Tsubu Anpan, is read from the ID memory 34 (step 4). Next, the read ID is checked against the main memory 30, and the characteristic part data (characteristic parts of the packaging bag, cross-sectional photos, etc. that differentiate the target product from other types of Anpan) and attribute data of the target product linked to the ID are read from the main memory 30 (step 5). Then, the read characteristic part data and attribute data are temporarily stored (memorized) in the temporary storage memory 25 (step 6).

[0045] Next, as shown in Fig. 6B, a surrounding environment image signal indicating the state of the shelf, etc., is input from the first camera 46 (step 7), and then this surrounding environment image signal is compared with the feature data stored in the temporary storage memory 25 to determine whether the feature data exists in the surrounding environment image signal (step 8). If the presence is confirmed (i.e., if the determination is YES), the presence of the target product is recognized, and if the determination is NO, the process returns to step 7. This loop is repeated until the target product is recognized.

[0046] When the presence of the target product is recognized, in step 9, the output audio signal creation unit 27 creates an audio signal to notify this recognition, and then, when the target product detection unit 26 detects (recognizes) the target product, it determines the direction and approximate distance of the target product from shopper P based on the position, size, etc. of the object in the image, and creates and outputs a direction / distance signal, i.e., a primary detection signal (step 10).

[0047] Based on the above, the shopper P gradually approaches the target product, and from the primary detection signal, determines the distance to the target product and judges whether the distance to the target product is within a predetermined range (step 11). If the judgment is NO, the process returns to step 9. If the judgment is YES, the camera etc. is automatically switched to that of the wearable device (step 12).

[0048] The processor receives an image signal from the second camera 62, which is a target product imaging unit (step 13), and while confirming that the target product is present within the field of view of the second camera 62 (step 13), determines the distance based on the distance signal input from the distance sensor 64, creates a presence / absence and distance signal or a distance signal (step 4), and outputs the audio signal to the speaker 44 of shopper P (step 15).

[0049] Based on the above, user P gradually approaches the target product (steps 14 and 15 are repeated until the determination in step 16 becomes YES), and when shopper P touches or grabs the target product with his / her hand, the determination in step 16 becomes YES and the control ends. As a result, a visually impaired shopper can grasp the desired product, Tsubuanpan, without using his or her eyes, and place it in the shopping cart.

[0050] As described above, the automatic checkout operation for the prices of items placed in the shopping cart can be performed automatically according to the flowchart shown in Figure 7, using the price data in the attribute data of the items stored in the temporary storage memory.

[0051] For this reason, it is preferable that the cash registers installed in the store be POS registers or the like that have a memory unit for totaling and can exchange data wirelessly.Furthermore, although not shown in the figure, it is preferable that the shopping support system for the visually impaired and others is equipped with a confirmation device that can confirm that products are in the shopping cart.

[0052] The automatic checkout operation first checks in step 1 that an item has been added to the cart, then in step 2, reads the price of that item from temporary memory and sends it to the POS register. If the same shopper repeats the above steps, the POS register stores the price in its memory and adds up the prices of the items for all those transactions (calculate the total amount) (step 4) and stores the total amount (including consumption tax). Shoppers can complete payments by tapping their credit card at the POS register.

[0053] As described above, by using the device and system of the present invention, visually impaired people and others will be able to walk on the streets and shop in stores more easily and safely than before, without the help of an assistant. [Explanation of symbols]

[0054] 10 Guidance devices for the visually impaired 20 Processing Unit 22 processors 30 Main Memory 40 Headset 42 Mike 44 speakers 46 Camera 1 60 Wearable Devices 62 Second Camera 64 Distance Sensor

Claims

1. A guidance device for visually impaired people that can be easily used by users who are visually impaired, etc. a processing unit connected to the processor and having a main memory in which at least a feature portion of the appearance of a predetermined object is stored in advance in association with an ID of the object; a headset worn on the user's head; A wearable device is provided which is attached to the wrist of one arm of a user or in the vicinity thereof, The headset comprises: an object indicating means for outputting a name signal representing the name of the object input by a user to the processing unit; an information transmission means for transmitting information to a user at least by voice; a surrounding environment imaging unit that continuously captures images of the user's surroundings and transmits surrounding environment image signals to the processing unit; The wearable device is an object imaging unit for continuously imaging an object and outputting an object image signal; a distance sensor that measures the distance between the object imaged by the object image capturing unit and a specific part of the user and outputs a distance signal; The processing unit an ID memory that stores an ID for each of the objects; an ID determination unit that receives a name signal from the object specifying means and compares the name signal with the ID memory to read and determine the ID of the object; a temporary storage memory for checking the determined ID against the main memory, reading the characteristic part of the object from the main memory, and temporarily storing the characteristic part; an object detection unit that detects the presence and position of an object in the surrounding environment by comparing the surrounding environment image signal from the surrounding environment imaging unit with the characteristic parts stored in the temporary storage memory; an output audio signal creation unit connected to the object detection unit, which creates audio data representing audio related to the detection of the object in response to an instruction from the object detection unit, and outputs the audio data to the information transmission means of the headset; the object detection unit has a function of specifying the direction of the object from the user based on the position of the object in the image when the object is detected, while the output audio signal creation unit receives the direction of the object from the object detection unit, creates a primary detection signal indicating the direction, and outputs the primary detection signal to the information transmission means; The processor receives an image signal from the object imaging unit, confirms whether an object is present within the field of view of the object imaging unit, and generates presence / absence and distance signals based on the distance signal from the distance sensor, and transmits them to the output audio signal creation unit. The output audio signal creation unit creates presence / absence and distance audio signals based on the received presence / absence and distance signals, and outputs them to the information transmission means. A guidance device for visually impaired people, etc.

2. A guidance device for visually impaired persons, etc. as claimed in claim 1, wherein the object detection unit has the function of, when detecting an object, determining the object's position in the image as well as its size and direction from the user and its approximate distance, while the output audio signal creation unit receives the object's direction and approximate distance from the object detection unit, creates a primary detection signal indicating the direction and approximate distance, and outputs this primary detection signal indicating the direction and approximate distance to the information transmission means.

3. 2. A guidance device for visually impaired persons according to claim 1, wherein the surrounding environment imaging unit is incorporated not in the headset but in a pouch or the like worn by the user.

4. 2. A guidance device for visually impaired persons as set forth in claim 1, wherein said object indicating means is a microphone for giving instructions by voice.

5. A shopping support system for visually impaired persons, etc., comprising the guidance device for visually impaired persons, etc. of claim 1, when a shopper who is a user of the device, etc., purchases products arranged on shelves at a store such as a convenience store, The ID memory stores in advance product IDs that identify products handled at the store, The processing unit is located in a store or in a shopping cart used by a shopper; the main memory stores at least characteristic portions of the external appearance of the product itself or the external appearance of the product's package, and attributes of the product, linked to a product ID of the product; the object specifying means is a microphone, and outputs a product name signal representing the name of the target product input by the shopper to the processing unit; the ID determination unit receives the product name signal from the object indicating means, checks the product name signal against the ID memory, and determines the product ID of the product represented by the product name signal; the temporary storage memory checks the product ID determined by the ID determination unit against the main memory, and temporarily stores at least a characteristic portion of the appearance form associated with the product ID based on the product ID; the surrounding environment imaging unit is provided in the shopping cart, and captures images of the external appearance of the products or the external appearance of the packages placed on the product shelves, and transmits image signals representing the images of the external appearances to the processing unit; the object detection unit compares the transmitted image signal with at least the characteristic portion stored in the temporary storage memory, and when a match is found, notifies the shopper of the recognition of the target product together with the general direction from the shopper; the object direction / distance specifying means specifies the direction and distance of the object relative to the user, and outputs an object direction / distance signal; A shopping support system for visually impaired persons, etc., characterized in that the processing unit receives a positional relationship signal output by the object direction / distance identification means and informs the shopper of the direction and distance to the target product indicated by the positional relationship signal.

6. 6. The shopping support system for visually impaired persons, etc., according to claim 5, wherein the main memory further stores the price of each product in association with the product ID.

7. 7. The shopping support system for visually impaired persons and the like according to claim 6, wherein said main memory reads out the price of the product from said storage unit for each identified product ID and transmits the price to a cash register system of the store.

8. 6. The shopping support system for visually impaired persons, etc., according to claim 5, wherein the main memory further stores a 3D image of the product itself or the external appearance of the product's packaging, linked to the product ID.

9. 6. A shopping support system for visually impaired persons, etc., according to claim 5, wherein when the target product is not within the field of view of the surrounding environment imaging unit and the target imaging unit, the object detection unit transmits a message to that effect to the audio signal creation unit, and the audio signal creation unit generates an audio signal to inform the shopper that the target product is not within the field of view.

10. 6. A shopping support system for visually impaired persons and the like according to claim 5, wherein the surrounding environment imaging unit is a wide-angle camera such as a 360-degree camera.

11. The shopping support system for visually impaired persons, etc. of claim 5 further comprises an imaging unit height position adjustment device that adjusts the height position of the imaging unit so that the arrangement height of the target products can be adjusted depending on the height of the shelf on which the target products are arranged.

Images