Guidance devices and shopping support systems for the visually impaired.

Abstract

To provide relatively low-cost guidance devices and shopping support systems for visually impaired persons. [Solution] A user-friendly guidance device for visually impaired persons, comprising a processor, a processing unit connected to the processor having a main memory that pre-stores at least the characteristic features of a predetermined object's appearance linked to the object's ID, a headset worn on the user's head, and a wearable device worn on or near the user's wrist. The processor receives an image signal from a camera, which is an object imaging unit provided in the wearable device, confirms whether or not the object exists within the field of view of the object imaging unit, and similarly creates presence / absence and distance signals based on distance signals from a distance sensor provided in the wearable device, and informs the user of the contents through a speaker such as an earphone provided in the headset, thereby providing safe walking navigation.

Description

[Technical Field]

[0001] The present invention relates to a guidance device for visually impaired persons and the like, and a shopping support system for visually impaired persons and the like. More specifically, the present invention relates to a guidance device for visually impaired persons and the like that enables visually impaired persons and the like to walk and move safely on roads, for example, and to a shopping support system for visually impaired persons and the like that enables visually impaired persons to actually pick up products when shopping at supermarkets, convenience stores and the like. [Background Art]

[0002] As the above-mentioned guidance device for visually impaired persons and the like, particularly the shopping support system for visually impaired persons and the like, an infrared guidance system and a user guidance terminal device are proposed in Japanese Patent Application Laid-Open No. 2006-258591.

[0003] The infrared guidance system proposed in this publication includes one or both of: a left transmitter that emits infrared rays carrying individual information in a direction intersecting the front direction of a user from the left side with respect to the front direction of the user, and a right transmitter that emits infrared rays carrying individual information in a direction intersecting the front direction of the user from the right side with respect to the front direction of the user; an infrared receiving device that includes a front infrared light receiving part for receiving infrared rays from the front, a left infrared light receiving part for receiving infrared rays from the left side, and a right infrared light receiving part for receiving infrared rays from the right side, and has a shape of an article worn above the user's neck or a shape that can be attached to such an article; a signal processing device that generates a guidance signal based on the individual information carried by the infrared rays received by the front infrared light receiving part, the left infrared light receiving part or the right infrared light receiving part; and an audio output device that notifies guidance information by voice based on the guidance signal.

[0004] Furthermore, the user guidance terminal device is characterized by comprising an infrared receiving device that has a front infrared receiving unit for receiving infrared light from the front, a left infrared receiving unit for receiving infrared light from the left, and a right infrared receiving unit for receiving infrared light from the right, and is shaped like an item worn on the user's neck or above, or shaped like an item that can be attached to such an item; a signal processing device that creates a guidance signal based on the infrared light received by the front infrared receiving unit, the left infrared receiving unit, or the right infrared receiving unit; and an audio output device that announces guidance information in voice based on the guidance signal.

[0005] The infrared guidance system and user guidance terminal device configured as described above have the advantage that visually impaired individuals can obtain information about the stores on the left and right by transmitting information via infrared while facing forward. This technology could be used to create a device or system that guides visually impaired shoppers to the shelves where their desired products are displayed when they shop at stores such as convenience stores (hereinafter sometimes referred to as "convenience stores").

[0006] However, the aforementioned infrared guidance system and user guidance terminal device are limited to the above, making it difficult for visually impaired shoppers to easily pick up desired items and shop on their own without the assistance of an assistant. Furthermore, an infrared generator must be installed for each target object, which presents the problem of requiring extensive installation work and incurring significant costs.

[0007] It should be noted that while there is a device for the visually impaired proposed in European Patent Publication No. 002490155, this proposed device is not intended for the completely blind, but rather for visually impaired individuals who have some degree of vision. Therefore, the target users differ from those of the device of the present invention, and consequently, its structure and configuration are also significantly different. [Prior art documents] [Patent Documents]

[0008] [Patent Document 1] Japanese Patent Publication No. 2006-258591 [Patent Document 2] European Patent Publication No. 002490155 [Overview of the project] [Problems that the invention aims to solve]

[0009] The present invention aims to solve the problems of the above-mentioned infrared guidance system and user guidance terminal device, and to provide a relatively low-cost guidance device and shopping support system for the visually impaired, enabling shoppers such as visually impaired individuals to easily pick up desired products and shop independently without the assistance of an assistant. [Means for solving the problem]

[0010] The above issues can be resolved by the following guidance devices and shopping support systems for the visually impaired, etc. (1) to (11). (1) A user guide device for the visually impaired that can be easily used by visually impaired users, A processor, and a processing unit connected to this processor, having a main memory that pre-stores at least characteristic parts of the appearance of a predetermined object, associated with the object's ID. A headset worn on the user's head, It comprises a wearable device that is attached to the wrist or near the user's arm, The aforementioned headset, Object indicating means that outputs a name signal representing the name of the object input by the user to the processing unit, A means of transmitting information to the user, at least by voice, The system includes a surrounding environment imaging unit that continuously captures images of the user's surrounding environment and transmits the surrounding environment image signals to the processing unit. The wearable device is A target object imaging unit that continuously images the target object and outputs a target object image signal, This object imaging unit includes a distance sensor that measures the distance between the object imaged by the object and a specific part of the user's body, and outputs a distance signal. The aforementioned processing unit is An ID memory that stores the ID for each of the aforementioned objects, An ID determination unit receives a name signal from the object indicating means and reads and determines the ID of the object by comparing it with the ID memory, The determined ID is compared with the main memory, the characteristic part of the object is read from the main memory, and a temporary memory is used to temporarily store this characteristic part. A target object detection unit detects the presence and location of a target object in the surrounding environment by comparing the surrounding environment image signal from the surrounding environment imaging unit with the feature section stored in the temporary memory, This unit is connected to the object detection unit and, based on instructions from the object detection unit, is equipped with an output audio signal generation unit that generates audio data indicating the detection of an object and outputs it to the headset's information transmission means. The aforementioned processor further, The object detection unit has a function to determine the direction and approximate distance of an object from the user based on its position, size, etc., in the image when it detects an object. Meanwhile, the output audio signal generation unit receives the direction and approximate distance of the object from the object detection unit, generates a primary detection signal indicating these, and outputs it to the information transmission means. The processor receives an image signal from the object imaging unit, confirms whether or not the object is within the field of view of the object imaging unit, and creates presence / absence and distance signals based on the distance signal from the distance sensor, and transmits these to the output audio signal generation unit. The output audio signal generation unit then creates presence / absence and distance audio signals based on the received presence / absence and distance signals, and outputs these to the information transmission means. A guidance device for visually impaired persons, characterized by the following features. (2) The guidance device for visually impaired persons, etc., according to (1), wherein the surrounding environment imaging unit is a wide-angle video camera, and the target object imaging unit is a standard video camera or a narrow-angle camera. (3) The aforementioned guidance device for visually impaired persons, etc., according to (1), wherein the surrounding environment imaging unit is incorporated into a pouch or the like worn by the user, rather than into the headset. (4) The guidance device for visually impaired persons, etc., according to (1), wherein the object indication means is a microphone that gives instructions by voice. (5) A shopping support system for visually impaired persons, equipped with the guidance device for visually impaired persons described in (1) above, for when a shopper who is a visually impaired person makes a purchase of goods arranged on shelves in a store such as a convenience store, The aforementioned ID memory pre-stores product IDs that identify the products handled at that store. The processing unit is installed in a shopping cart used by the store or a shopper. The aforementioned main memory stores at least a characteristic part of the external appearance of the product itself or the external appearance of the product's packaging, associated with the product ID of the product. The aforementioned target item indicating means is a microphone that outputs a product name signal representing the name of the target product input by the shopper to the processing unit. The ID determination unit receives the product name signal from the target object indicating means, compares the product name signal with the ID memory, and determines the product ID of the product represented by the product name signal. The temporary storage memory compares the product ID determined by the ID determination unit with the main memory and temporarily stores at least the characteristic portion of the external form associated with that product ID. The surrounding environment imaging unit is installed in the shopping cart and captures images of the external appearance of products or the external appearance of their packaging placed on the product shelves, and transmits image signals showing these external appearances to the processing unit. Said target object detection unit compares the transmitted image signal against at least the feature portion stored in said temporary storage memory, and when a match is found, notifies the shopper of the recognition of the target product along with the approximate direction from the shopper, Said target object direction and distance specifying means specifies the direction and distance of the target object relative to the user, and outputs a target object direction and distance signal, Said processing unit receives the positional relationship signal output by said target object direction and distance specifying means, and is configured to notify the shopper of the direction and distance to the target product indicated by said positional relationship signal; which is a shopping assistance system for visually impaired persons and the like. (6) The shopping assistance system for visually impaired persons and the like according to (5) above, wherein said storage unit further stores the price of the product in association with the product ID. (7) The shopping assistance system for visually impaired persons and the like according to (6) above, wherein said processing unit reads the price of the product from said storage unit for each specified product ID, and transmits said price to the store's cashier system. (8) The shopping assistance system for visually impaired persons and the like according to (5) above, wherein said main memory further stores a 3D image of the external appearance of the product itself or the product's packaging in association with the product ID. (9) The shopping assistance system for visually impaired persons and the like according to (5) above, wherein when there is no target product within the field of view of said surrounding environment imaging unit and target object imaging unit, said target object detection unit transmits notification of this fact to an audio signal generation unit, and said audio signal generation unit generates an audio signal that notifies the shopper that the target product is not within the field of view. (10) The shopping assistance system for visually impaired persons and the like according to (5) above, wherein said surrounding environment imaging unit is a wide-angle camera such as a 360-degree camera. (11) The shopping support system for visually impaired persons, etc., further includes an imaging unit height adjustment device for adjusting the height of the imaging unit, so that it can accommodate the height of the shelves on which the target products are arranged. [Effects of the Invention]

[0011] In the above-described guide device and shopping support system for the visually impaired, the target object is captured using an imaging means, or camera, which acts as a substitute for the visually impaired person's vision. Therefore, it becomes possible to walk around town or shop by actually picking up products in stores without transmitting information indicating the attributes and content of each target object. [Brief explanation of the drawing]

[0012] [Figure 1] Figure 1 is a schematic diagram of a guidance device for visually impaired persons, etc., according to an embodiment of the present invention. [Figure 2] Figure 2 is a block diagram illustrating the functions of the guidance device for visually impaired persons shown in Figure 1. [Figure 3A] Figure 3A is a flowchart illustrating the operation of the control unit of a guidance device for visually impaired persons according to an embodiment of the present invention. [Figure 3B] Figure 3B is a flowchart illustrating the operation of the control unit of the guidance device for visually impaired persons according to an embodiment of the present invention, and is a continuation of Figure 3A. [Figure 4] Figure 4 is a system diagram of a shopping support system for visually impaired persons according to an embodiment of the present invention. [Figure 5] Figure 5 is a block diagram illustrating the functions of the shopping support system for visually impaired persons shown in Figure 4. [Figure 6A] Figure 6A is a flowchart illustrating the operation of the control unit of the shopping support system for visually impaired persons according to an embodiment of the present invention. [Figure 6B]Figure 6B is a flowchart illustrating the operation of the control unit of the shopping support system for visually impaired persons according to an embodiment of the present invention, and is a continuation of Figure 6A. [Figure 7] Figure 7 is a checkout flowchart illustrating the operation of the control unit of the shopping support system for visually impaired persons according to an embodiment of the present invention. [Modes for carrying out the invention]

[0013] Hereinafter, with reference to the attached drawings, a guide device 10 for visually impaired persons, etc., according to an embodiment of the present invention will be specifically described. In this specification, "visually impaired persons, etc." broadly refers to persons who are completely blind, as exemplified below, infants who cannot distinguish between objects, etc. Here are some examples of visual impairments. • Total blindness: A state of complete loss of sight. Because visual information cannot be used, Braille and audio guidance are important means of support. • Amblyopia: A condition in which vision is partially impaired. Vision correction devices (glasses or contact lenses), magnifying readers, and audio guidance are effective. • Visual field constriction: A condition in which the field of vision narrows. In some cases, the central vision may be preserved, while in others, the peripheral vision may be preserved. Examples include tunnel vision (central vision is preserved and peripheral vision is lost) and peripheral vision loss (peripheral vision is preserved and central vision is lost). • Night blindness: A condition in which vision is impaired at night or in dark places. While vision may be fine in bright places, vision may be almost completely impaired in dark places. Other: Individuals with extreme nearsightedness or extreme farsightedness (presbyopia). In this specification, such visually impaired individuals may be referred to as users.

[0014] This guidance device 10 for visually impaired persons is a device that can be easily used by a user P who is visually impaired, and comprises a processing unit 20 that is worn by the user P like a waist pouch, a headset 40 that is worn on the user P's head, and a wearable device 60 that is worn on the wrist of one of the user P's arms or near it. At least a part of the processing unit 20 may be provided in the headset 40, etc.

[0015] The processing unit 20 includes a processor 22 for performing various calculations and processing to guide and support the user P, and a main memory 30 connected to the processor 22 that stores feature data representing at least the feature parts of the appearance of a predetermined object, linked to the object's ID. If the object is a push-button traffic signal S as shown in Figure 1, it includes a push-button box PB, and a circular push button pb for the user to operate is placed at a predetermined location on the surface of the push-button box PB. Here, the feature parts of the object are the configuration of the push button pb, namely, shape: circular, color: red, position: lower part of the push-button box PB, center in the left-right direction, etc., and this data is stored in the main memory, linked to the object's ID.

[0016] The headset 40 includes a microphone 42, which is an object indicating means that outputs a name signal representing the name of the object input by the user to the processing unit 20, The system includes a speaker 44, such as an earphone, as a means of transmitting information to the user, at least by voice, and a first camera 46, such as a 360-degree camera or wide-angle camera, as a peripheral environment imaging unit that continuously captures images of the user's surroundings and transmits peripheral environment image signals to the processing unit.

[0017] The wearable device 60 includes a second camera 62, which is an object imaging unit that continuously images an object at a closer distance than the first camera 46 and outputs an object close-range image signal, and a distance sensor 64 that measures the distance between the object imaged by the second camera 62 and a specific part of the user, for example, the second camera 62 or the hand at the end of the wrist of the user P to which the wearable device 60 is attached, and outputs a distance signal. The second camera 62 is preferably a camera with a normal field of view or a narrow-angle camera.

[0018] The processing unit 20 further includes an input / output interface 32 for inputting and outputting signals between the processor 22 and the headset 40 or wearable device 60, and an ID memory 34 connected to the processor 22 that stores an ID for each target object.

[0019] The processor 22 includes an ID determination unit 23 that receives a name signal from the microphone 42 and reads and determines the ID of the target object by comparing it with the ID memory 34; a reading unit 24 that compares the determined ID with the main memory and reads the feature data of the target object from the main memory; and a temporary storage memory 25 that temporarily stores the feature data read by the reading unit 24.

[0020] The processor 22 further includes an object detection unit 26 that detects the presence and location of an object in the surrounding environment by comparing the surrounding environment image signal from the first camera 46 with the feature data stored in the temporary memory 25, and an output audio signal generation unit 27 connected to the object detection unit 26, which generates audio data indicating the detection of an object based on instructions from the object detection unit 26 and outputs it to the speaker 44 of the headset 40.

[0021] Preferably, the object detection unit 26 has a function to determine the direction and approximate distance of an object from the user P based on the position and size of the object in the image when it detects an object. A distance sensor may be provided separately for distance. The output audio signal generation unit 27 generates a primary detection signal from the object detection unit 26 indicating the direction and approximate distance of the object, and outputs this to the speaker 44 of the headset 40. The guidance device 10 for visually impaired persons has a switching means (not shown) that switches the means for outputting to the processing unit 20 from the headset 40 to the wearable device 60 when the user P determines that the direction and distance of the object have been determined from the audio emitted from the speaker 44 based on the primary detection signal and that the distance of the object has fallen within a predetermined range. This switching means may be capable of automatically performing the switch when the distance indicated by the primary detection signal falls within a predetermined range.

[0022] The processor further includes an object presence confirmation unit 28 that receives an image signal from the second camera 62, which is an object imaging unit, checks whether or not an object exists within the field of view of the second camera 62, and determines the distance based on the distance signal from the distance sensor 64, thereby creating an presence / absence and distance signal. The object presence confirmation unit 28 transmits this created signal to the output audio signal creation unit, which then creates an presence / absence and distance audio signal based on the received presence / absence and distance signal, and outputs it to the speaker 44.

[0023] Next, the operation of the processing unit 20 and the processor 22 will be explained using the flowcharts in Figures 3A and 3B. Note that this flowchart primarily describes the operation from the perspective of the processor 22.

[0024] First, as shown in the flowchart in Figure 3A, the headset 40 receives a target object name signal from the microphone 42 (step 1). This signal is then compared with the ID memory 34 to search for an ID that matches the name signal (step 2). If the result is NO, a prompt signal is output to the user P to prompt them to provide a new target object signal (step 3).

[0025] On the other hand, if the above determination is YES, the ID is determined and the ID of the object is read from the ID memory 34 (step 4). Next, the read ID is compared with the main memory 30 and the characteristic data of the object associated with that ID (such as the shape characteristics of the push button box, the shape characteristics of the push button (circular), the positional relationship in the push button box, etc.) is read from the main memory 30 (step 5). Then, the read characteristic data is temporarily stored in the temporary storage memory 25 (step 6).

[0026] Next, as shown in Figure 3B, the ambient image signal from the first camera 46 is input (step 7). Then, this ambient image signal is compared with the feature data stored in the temporary memory 25 to determine whether or not the feature data exists in the ambient image signal (step 8). If its presence is confirmed (i.e., the determination is YES), the presence of the target object is recognized (step 9). If the determination is NO, the process returns to step 7. This loop is repeated until the target object is recognized.

[0027] When the presence of the target object is recognized (step 9), in step 10, the output audio signal generator 27 creates an audio signal to indicate this recognition. Then, when the target object detection unit 26 detects the target object, it determines the direction and approximate distance of the target object from the user P based on the position and size of the target object in the image, and creates and outputs a direction / distance signal, i.e., a primary detection signal (step 11).

[0028] Based on the above, user P gradually approaches the target object, determines the distance to the target object using the primary detection signal, and determines whether the distance to the target object is within a predetermined range (step 12). If the determination is NO, the user returns to step 9; if YES, the camera, etc., is automatically switched to that of the wearable device (step 13). This switching can be configured in any way.

[0029] The processor receives an image signal from the second camera 62, which is the object imaging unit (step 14), confirms that the object is within the field of view of the second camera 62, determines the distance based on the distance signal from the distance sensor 64, creates an existence / non-existence and distance signal or distance signal, and outputs the audio signal to the speaker 44 of the user P (step 15).

[0030] Based on the above, user P gradually approaches the target object (repeating steps 14 and 15 until the judgment in step 16 is YES), and when user P's hand (fingers) touches the push button of the traffic light, i.e., when P presses it, the judgment in step 16 becomes YES, and the control ends. As a result, the user can press the push button on the traffic light, which is the target object, without using their eyes.

[0031] Next, with reference to Figure 4 and subsequent figures, a shopping support system 100 for visually impaired persons according to an embodiment of the present invention will be described in detail. In this description of the shopping support system 100 for visually impaired persons, components and parts having the same or similar functions as the guide device 10 for visually impaired persons will be denoted by the same reference numerals. In this shopping support system 100 for visually impaired persons, the processing unit 20 consists of a PC or the like, which is installed in the store itself or in a shopping cart C provided in the store. The processing unit 20 may have a structure like a waist pouch, similar to the guidance device for visually impaired persons. The headset 40 and wearable device 60 are worn on the head of a visually impaired shopper, on the wrist of the hand that the visually impaired shopper actually uses to grasp the goods, etc.

[0032] The processing unit 20 includes a processor 22 for performing various calculations and processing to guide and support the user (shopper) P during their shopping, and a main memory 30 connected to the processor 22 that stores feature data representing at least the distinctive appearance of the target product G or its packaging (bag, box, etc.) and the attributes of the target product, linked to the ID of the target product.

[0033] In this context, the attributes of the target product include the height of the shelf where the product is placed, its price, and promotional information for that product (discount sales, point multipliers, etc.).

[0034] Similar to the guidance device 10 for the visually impaired, the headset 40 includes a microphone 42, which is a target product instruction means that outputs a name signal representing the name of the target product (for example, Anpan) input by the shopper P to the processing unit 20, and a speaker 44 such as an earphone, which is an information transmission means for transmitting information to the user, at least by voice. Furthermore, this headset 40 may also include the first camera found in the guidance device 10 for the visually impaired described above.

[0035] The shopping cart C is equipped with a first camera 46, such as a 360-degree camera or wide-angle camera, which serves as a surrounding environment imaging unit that continuously captures images of the user's surrounding environment and transmits the surrounding environment image signals to the processing unit. Preferably, this first camera 46 is mounted on a support column of the shopping cart C so as to be vertically movable and automatically adjusts to the height of the shelf on which the product is placed, as stored as an attribute of the target product. With this first camera 46, shoppers, such as those with visual impairments, can find the target product while walking through the aisles between product display shelves.

[0036] The wearable device 60 includes a second camera 62, which is a target product imaging unit that continuously images the target product at a closer distance than the first camera 46 and outputs a close-range image signal of the target product, and a distance sensor 64 that measures the distance between the target product imaged by the second camera 62 and a specific part of the user, for example, the second camera 62 or the hand at the end of the wrist of the user P to which the wearable device 60 is attached, and outputs a distance signal. The second camera 62 is preferably a camera with a normal field of view or a narrow-angle camera.

[0037] The processing unit 20 further includes an input / output interface 32 for inputting and outputting signals between the processor 22 and the headset 40 or wearable device 60, and an ID memory 34 connected to the processor 22 that stores an ID for each target product.

[0038] The processor 22 includes an ID determination unit 23 that receives a name signal of the target product from the microphone 42 and reads and determines the ID of the target product by comparing it with the ID memory 34; a reading unit 24 that compares the determined ID with the main memory and reads the feature data and attribute data of the target product from the main memory; and a temporary storage memory 25 that temporarily stores the feature data and attribute data read by the reading unit 24.

[0039] The processor 22 further includes a target product detection unit 26 that detects the presence and location of a target product in the surrounding environment by comparing the surrounding environment image signal from the first camera 46 with the feature data stored in the temporary memory 25, and an output audio signal generation unit 27 connected to the target product detection unit 26, which generates audio data indicating the detection of the target product based on instructions from the target product detection unit 26 and outputs it to the speaker 44 of the headset 40. Here, data such as whether the product is a discount product among the product attribute data may also be read from the temporary memory 25 at the same time, and the information indicated by this data may be included in the audio data.

[0040] Preferably, the target product detection unit 26 has a function to determine the direction and approximate distance of the target product from the user P based on the position, size, etc., of the target product in the image when it detects the target product. The output audio signal creation unit 27 creates a primary detection signal from the target product detection unit 26 indicating the direction and approximate distance of the target product, and outputs this to the speaker 44 of the headset 40. The shopping guidance device 100 for visually impaired persons has a switching means (not shown) that switches the means for outputting to the processing unit 20 from the shopping cart C to the wearable device 60 when the user P determines that the direction and distance of the target product have been determined from the audio emitted from the speaker 44 based on the primary detection signal and that the distance of the target product is within a predetermined range. This switching means may be capable of automatically performing the switch when the distance indicated by the primary detection signal is within a predetermined range.

[0041] The processor, upon switching as described above, now receives an image signal from the second camera 62, which is the target product imaging unit, and confirms whether or not the target product is present within the field of view of the second camera 62. It also determines the distance based on the distance signal from the distance sensor 64 and further includes a target product presence confirmation unit 28 that creates presence / absence and distance signals. The target product presence confirmation unit 28 transmits the created signals to the output audio signal creation unit, which then creates presence / absence and distance audio signals based on the received presence / absence and distance signals and outputs them to the speaker 44.

[0042] Next, the operation of the processing unit 20 and the processor 22 will be explained using the flowcharts in Figures 6A and 6B. Note that this flowchart explanation is primarily from the perspective of the processor 22.

[0043] First, as shown in the flowchart in Figure 6A, the headset 40 receives a target product instruction signal, which is the name of the target product (here, we will explain assuming the target product is a sweet bean paste bun with chunks of bean paste) from the microphone 42 (Step 1). This is then compared with the ID memory 34 to search for an ID that matches the name signal and make a determination (Step 2). If this determination is NO, a prompt signal is output to the user P prompting them to give a new target product instruction signal, and this is repeated until the determination is YES (Step 3). If the store sells two types of sweet bean paste buns, "sweet bean paste bun with chunks of bean paste" and "smooth sweet bean paste bun," and the instruction is simply "sweet bean paste bun," the shopper is informed of the above and prompted to give an instruction again.

[0044] On the other hand, if the above determination is YES, the ID is determined and the ID of the target product, Tsubu Anpan, is read from the ID memory 34 (Step 4). Next, the read ID is compared with the main memory 30 and the feature data (features of the packaging bag, photographs of cross-sections, etc., that differentiate it from other types of Anpan) and attribute data of the target product associated with that ID are read from the main memory 30 (Step 5). Then, the read feature data and attribute data are temporarily stored in the temporary storage memory 25 (Step 6).

[0045] Next, as shown in Figure 6B, an ambient image signal from the first camera 46 showing the state of the shelf, etc., is input (step 7). Then, this ambient image signal is compared with the feature data stored in the temporary memory 25 to determine whether or not the feature data exists in the ambient image signal (step 8). If its presence is confirmed (i.e., the determination is YES), the presence of the target product is recognized. If the determination is NO, the process returns to step 7. This loop is repeated until the target product is recognized.

[0046] When the presence of the target product is recognized, in step 9, the output audio signal generator 27 creates an audio signal to indicate this recognition. Then, when the target product detection unit 26 detects (recognizes) the target product, it determines the direction and approximate distance of the target product from the shopper P based on the position and size of the target object in the image, and creates and outputs a direction / distance signal, i.e., a primary detection signal (step 10).

[0047] Based on the above, shopper P gradually approaches the target product, determines the distance to the target product using the primary detection signal, and determines whether the distance to the target product is within a predetermined range (step 11). If the determination is NO, the process returns to step 9; if YES, the camera, etc., is automatically switched to that of the wearable device (step 12).

[0048] The processor receives an image signal from the second camera 62, which is the target product imaging unit (step 13), confirms that the target product is within the field of view of the second camera 62 (step 13), determines the distance based on the distance signal from the distance sensor 64, creates an audio signal indicating presence or absence and distance (step 4), and outputs the audio signal to the shopper P's speaker 44 (step 15).

[0049] Based on the above, user P gradually approaches the target product (repeating steps 14 and 15 until the judgment in step 16 is YES), and when shopper P touches or grasps the target product with their hand, the judgment in step 16 becomes YES, and control ends. As a result, shoppers with visual impairments can grasp the desired product, the red bean bun, without using their eyes and place it in their shopping cart.

[0050] As described above, the automated checkout process for the prices of items placed in the shopping cart can be performed automatically using the price data in the attribute data of the items stored in the temporary memory, following the flowchart shown in Figure 7.

[0051] Therefore, the cash register installed in the store is preferably a POS register or similar device equipped with a memory unit for totaling and capable of wirelessly exchanging data. Furthermore, although not shown in the illustration, it is preferable that this shopping support system for visually impaired persons is equipped with a confirmation device that can verify that there are items in the shopping cart.

[0052] The automated checkout process begins in step 1, confirming that an item has been placed in the cart. Then, in step 2, it reads the price of that item from temporary memory and transmits it to the POS register. When the same customer repeats the above steps, the POS register stores these items in its memory unit and performs an operation to add up the prices of the items for each transaction (calculation of the total amount) (step 4), and stores the total amount (including consumption tax). Shoppers can complete their payments using contactless payment methods such as credit card payment at the aforementioned POS register.

[0053] As described above, by using the device and system of the present invention, visually impaired persons and others will be able to walk on streets and shop in stores more easily and safely than before, without the assistance of an assistant. [Explanation of Symbols]

[0054] 10. Guidance devices for visually impaired persons, etc. 20 processing units 22 processors 30 Main Memory 40 Headsets 42 Mike 44 speakers 46. ​​Camera 1 60 Wearable Devices 62. Second camera 64 Distance Sensors

Claims

1. A user guide device for the visually impaired that can be easily used by visually impaired users, A processor, and a processing unit connected to this processor, having a main memory that has pre-stored at least characteristic parts of the appearance of a predetermined object, associated with the ID of that object; A headset worn on the user's head, It comprises a wearable device that is attached to the wrist or near the user's arm, The aforementioned headset, Object indicating means that outputs a name signal representing the name of the object input by the user to the processing unit, A means of transmitting information to the user, at least by voice, The system includes a surrounding environment imaging unit that continuously captures images of the user's surrounding environment and transmits the surrounding environment image signals to the processing unit. The wearable device is A target object imaging unit that continuously images the target object and outputs a target object image signal, This object imaging unit includes a distance sensor that measures the distance between the object imaged by the object and a specific part of the user's body, and outputs a distance signal. The aforementioned processing unit is An ID memory storing the ID for each of the aforementioned objects, An ID determination unit receives a name signal from the object indicating means and reads and determines the ID of the object by comparing it with the ID memory, The determined ID is compared with the main memory, the characteristic part of the object is read from the main memory, and a temporary memory is used to temporarily store this characteristic part. A target object detection unit detects the presence and location of a target object in the surrounding environment by comparing the surrounding environment image signal from the surrounding environment imaging unit with the feature section stored in the temporary storage memory, This unit is connected to the object detection unit and, based on instructions from the object detection unit, is equipped with an output audio signal generation unit that generates audio data indicating the detection of an object and outputs it to the headset's information transmission means. The aforementioned processor further, The object detection unit has a function to determine the direction and approximate distance of an object from the user based on its position and size in the image when it detects an object. Meanwhile, the output audio signal generation unit receives the direction and approximate distance of the object from the object detection unit, generates a primary detection signal indicating these, and outputs it to the information transmission means. The processor receives an image signal from the object imaging unit, confirms whether or not the object is within the field of view of the object imaging unit, and creates presence / absence and distance signals based on the distance signal from the distance sensor, and transmits these to the output audio signal generation unit. The output audio signal generation unit then creates presence / absence and distance audio signals based on the received presence / absence and distance signals, and outputs these to the information transmission means. A guidance device for visually impaired persons, characterized by the following features.

2. The guide device for visually impaired persons, etc., according to claim 1, wherein the surrounding environment imaging unit is a wide-angle video camera and the target object imaging unit is a standard video camera or a narrow-angle camera.

3. The guide device for visually impaired persons, etc., according to claim 1, wherein the surrounding environment imaging unit is incorporated into a pouch or the like worn by the user, rather than into the headset.

4. The guidance device for visually impaired persons, etc., according to claim 1, wherein the object indication means is a microphone that gives instructions by voice.

5. A shopping support system for visually impaired persons, comprising a guidance device for visually impaired persons, etc. according to claim 1, for when a shopper who is a visually impaired person, etc., makes a purchase of goods arranged on shelves in a store such as a convenience store, The aforementioned ID memory pre-stores product IDs that identify the products handled at that store. The processing unit is installed in a shopping cart used by the store or a shopper. The main memory stores at least characteristic parts of the external appearance of the products handled by the store or the external appearance of the product's packaging, and attributes of those products, linked to the product ID of each product. The aforementioned target item indicating means is a microphone that outputs a product name signal representing the name of the target product input by the shopper to the processing unit. The ID determination unit receives the product name signal from the target object indicating means, compares the product name signal with the ID memory, and determines the product ID of the product represented by the product name signal. The temporary storage memory compares the product ID determined by the ID determination unit with the main memory and temporarily stores at least the characteristic portion of the external form associated with that product ID. The surrounding environment imaging unit is installed in the shopping cart and captures images of the external appearance of products or the external appearance of their packaging placed on the product shelves, and transmits image signals showing these external appearances to the processing unit. The object detection unit compares the transmitted image signal with at least the feature portion stored in the temporary memory, and when a match is found, it notifies the shopper of the recognition of the target product, along with its general direction from the shopper. The aforementioned object direction and distance identification means identifies the direction and distance of the object relative to the user and outputs an object direction and distance signal. The shopping support system for visually impaired persons, characterized in that the processing unit receives a positional relationship signal output by the object direction and distance identification means and informs the shopper of the direction and distance to the target product indicated by the positional relationship signal.

6. The shopping support system for visually impaired persons, etc., according to claim 5, wherein the memory unit further stores the price of the product, linked to the product ID.

7. The shopping support system for visually impaired persons, etc., according to claim 6, wherein the processing unit reads the price of a product from the storage unit for each identified product ID and transmits the price to the store's cash register system.

8. The shopping support system for visually impaired persons according to claim 5, wherein the main memory further stores a 3D image of the product itself or the external appearance of the product's packaging, linked to the product ID.

9. The shopping support system for visually impaired persons, etc., according to claim 5, wherein the target object detection unit transmits to the audio signal generation unit when the target product is not within the field of view of the surrounding environment imaging unit and the target object imaging unit, and the audio signal generation unit generates an audio signal to inform the shopper that the target product is not within the field of view.

10. The shopping support system for visually impaired persons, etc., according to claim 5, wherein the surrounding environment imaging unit is a wide-angle camera such as a 360-degree camera.

11. The shopping support system for visually impaired persons, etc., further comprises an imaging unit height adjustment device for adjusting the height position of the imaging unit, so as to be able to accommodate the height of the arrangement of the target products, such as the height of the shelf on which the target products are arranged, according to claim 5.

Images