Electronic equipment and information processing systems

Abstract

A power generation unit 30 generates power. A power storage unit 32 stores power obtained in the power generation unit 30. An image sensor 16 captures an image of the circumference of an electronic device. A display unit 12 displays an image corresponding to a subject captured by the image sensor 16. The image sensor 16 and the display unit 12 are driven by the power stored in the power storage unit 32.

Description

Technical Field

[0001] The present disclosure relates to an electronic device and an information processing technology using the electronic device.

Background Art

[0002] Patent Document 1 discloses an information processing system including an information processing device, a cart, and a mat. The cart includes an image sensor (camera) that captures an array pattern printed on the mat, and acquires position information and control information on the mat by reading the array pattern. The information processing device acquires the position information and the control information from the cart and controls the operation of the cart. The cart is equipped with a battery that can be charged by an external power source and is driven by the power supplied from the battery.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] For an electronic device using a battery, when the remaining battery level runs out, battery replacement or charging is required. Therefore, if there is no replaceable battery, or if there is no charger or external power source for charging the battery, the power of the electronic device cannot be ensured and the electronic device cannot be operated.

[0005] The present disclosure has been made in consideration of the above circumstances, and an object thereof is to provide a technology related to an electronic device that does not mount a secondary battery or a primary battery (a disposable type battery) charged by an external power source.

Means for Solving the Problems

[0006] To solve the above problems, an electronic device according to one aspect of the present invention comprises a power generation unit that generates electricity, a power storage unit that stores the electricity obtained from the power generation unit, an image sensor that captures images of the area around the electronic device, and a display unit that displays an image corresponding to the subject captured by the image sensor. The image sensor and the display unit are driven by the electricity stored in the power storage unit.

[0007] Another aspect of the present invention is an information processing system comprising an electronic device and an information processing device having a camera for photographing the electronic device, wherein the electronic device comprises a power generation unit for generating electricity, a power storage unit for storing the electricity obtained from the power generation unit, and a first display unit that is driven by the electricity stored in the power storage unit to display an image, and the information processing device comprises a recognition unit for recognizing an image captured by the camera and a second display unit that displays an image corresponding to the result recognized by the recognition unit.

[0008] Any combination of the above components, as well as conversions of the expression of the present invention between methods, apparatus, systems, computer programs, recording media on which computer programs are recorded in a readable manner, data structures, etc., are also valid embodiments of the present invention. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows the electronic equipment according to the embodiment arranged on a mat. [Figure 2] This is a diagram showing the front view of an electronic device. [Figure 3] This is a diagram showing a side view of an electronic device. [Figure 4] This is a diagram showing the bottom view of an electronic device. [Figure 5] This is a diagram showing the functional blocks of an electronic device. [Figure 6] This figure shows an example of the operation flow of each component in an electronic device. [Figure 7] This figure shows the display unit with an object image shown. [Figure 8] This figure shows an example of an information processing system that includes electronic equipment and information processing equipment. [Figure 9] This diagram shows the functional blocks of an information processing device. [Modes for carrying out the invention]

[0010] Figure 1 shows the electronic device 10 according to the embodiment placed on the mat 2. The electronic device 10 in the embodiment may be a toy that a user can hold and play with. The mat 2 is divided into a plurality of grid-like regions, and a code is printed or attached to each region. In the embodiment, the code may include at least positional information indicating the location of the region on the mat 2 and information about the object associated with that region (hereinafter referred to as "object information"). The type of code may be a one-dimensional code or a two-dimensional code, or it may be human-readable text as shown in the figure. In any case, the code is composed of information that the electronic device 10 can read.

[0011] The electronic device 10 comprises a housing 20 having a polyhedral shape. In the embodiment, the housing 20 is preferably sized so that a user can grasp it with one hand. The housing 20 may have a rectangular parallelepiped shape with a height of about 2 to 3 cm and a length of about 3 to 5 cm on each side, and the length and width may be the same. The electronic device 10 has a rectangular display unit 12 on its top surface and an image sensor on its bottom surface that captures the codes attached to each area of ​​the mat 2. In the embodiment, when the image sensor captures a code, the display unit 12 displays an image corresponding to the object information contained in the captured code. The displayed image may be an image representing an object.

[0012] The electronic device 10 in this embodiment may have a driving function. The electronic device 10 drives on the mat 2 and stops within a partitioned area. For example, if the code contained in each area contains destination information indicating the location of the next area to proceed to, the electronic device 10 may autonomously drive to the next area by reading the destination information contained in the code. The electronic device 10 may also drive on the mat 2 in accordance with driving instructions from an external control device (not shown).

[0013] Figure 2 shows a front view of the electronic device 10, Figure 3 shows a side view of the electronic device 10, and Figure 4 shows a bottom view of the electronic device 10. The electronic device 10 of this embodiment has two wheels 14a and 14b (hereinafter referred to as "wheels 14" unless otherwise specified) on the bottom of the housing 20. A motor is connected to each wheel 14.

[0014] The image sensor 16 is a camera that captures images of the area around the electronic device 10 and is located on the bottom surface of the housing 20. Therefore, the image sensor 16 captures images of the area below the housing 20. The image sensor 16 may also be provided on the front, right side, back, left side, top, etc. of the housing 20. The electronic device 10 in this embodiment has the function of analyzing the image captured by the image sensor 16 to identify the subject and displaying the image corresponding to the subject on the display unit 12. Specifically, the electronic device 10 identifies the code contained in the captured image and displays the image corresponding to the object information contained in the code on the display unit 12.

[0015] The lighting unit 18 has an LED that lights up and is located on the bottom surface of the housing 20. The lighting unit 18 is provided so as to protrude downward from the bottom surface of the housing 20. When the electronic device 10 is placed on the mat 2 with its top surface facing upward (bottom surface facing downward), the wheels 14a, 14b and the lighting unit 18 make contact with the ground. The electronic device 10 can move on the mat 2 by rotating the wheels 14a and 14b.

[0016] Figure 5 shows the functional blocks of the electronic device 10. The electronic device 10 includes a display unit 12, wheels 14, an image sensor 16, a lighting unit 18, a power generation unit 30, a power storage unit 32, an inertial measuring device (hereinafter referred to as "IMU") 34, a control unit 36, and a communication unit 42. The IMU 34 may be an attitude sensor including a 3-axis accelerometer and a 3-axis gyroscope. The control unit 36 ​​includes a first control unit 38 and a second control unit 40.

[0017] The electronic device 10 includes a computer, and various functions are realized by the computer executing a program. The computer includes, as hardware, a memory for loading the program, one or more processors for executing the loaded program, an auxiliary storage device, and other LSIs. The processor is composed of a plurality of electronic circuits including semiconductor integrated circuits and LSIs, and the plurality of electronic circuits may be mounted on one chip or may be mounted on a plurality of chips. The functional blocks shown in FIG. 5 are realized by the cooperation of hardware and software. Therefore, it is understood by those skilled in the art that these functional blocks can be realized in various forms by only hardware, only software, or a combination thereof.

[0018] The electronic device 10 of the embodiment does not include a secondary battery charged by an external power source, for example, a commercial power source, or a primary battery. In the electronic device 10 of the embodiment, the power generation unit 30 has a power generation function and generates power used in the electronic device 10. The power storage unit 32 stores the power obtained by the power generation unit 30, and the display unit 12, the image sensor 16, the lighting unit 18, the IMU 34, the control unit 36, and the communication unit 42 are driven by the power stored in the power storage unit 32.

[0019] FIG. 6 shows an example of the operation flow of each component in the electronic device 10. In the embodiment, the power generation unit 30 is a motor that generates power in conjunction with the rotation of the wheel 14. The user holds the electronic device 10 by hand and rotates the wheel 14, so that the motor generates power (S10). When the user moves the electronic device 10 back and forth with the wheel 14 attached to the floor or table and rotates the wheel 14, the motor connected to the wheel 14 generates power. The power storage unit 32 is composed of a capacitor or the like and stores the generated power (S12).

[0020] When power is stored in the energy storage unit 32, the first control unit 38 is immediately activated (S14) and monitors the amount of energy stored in the energy storage unit 32 (S16). The first control unit 38 is a microcontroller and controls the activation of the second control unit 40 according to the amount of energy being monitored. The first control unit 38 may monitor the amount of energy stored in the energy storage unit 32 by monitoring the voltage of the energy storage unit 32.

[0021] If the amount of charge stored in the power storage unit 32 is less than a predetermined value (N in S16), the first control unit 38 waits for the user to rotate the wheel 14 further to increase the amount of charge. When the amount of charge stored in the power storage unit 32 exceeds the predetermined value (Y in S16), the first control unit 38 lights up the indicator light 18 for a predetermined time (for example, 0.5 seconds) (S18), and supplies power from the power storage unit 32 to the second control unit 40 to start up the second control unit 40 (S20). Preferably, the first control unit 38 supplies the power stored in the power storage unit 32 to the second control unit 40 only if the amount of charge stored in the power storage unit 32 has reached a predetermined amount that allows the second control unit 40 to operate stably. The user recognizes that sufficient power has been stored in the electronic device 10 by confirming that the indicator light 18 is lit. Note that the lighting control of the indicator light 18 may be performed by the started second control unit 40.

[0022] The second control unit 40 receives power from the energy storage unit 32, enabling it to control the operation of the display unit 12, image sensor 16, IMU 34, and communication unit 42. Immediately after startup, the second control unit 40 may operate in a low-power consumption mode to reduce power consumption. In low-power consumption mode, the second control unit 40 may reduce power consumption by not driving at least the image sensor 16. Furthermore, in low-power consumption mode, the second control unit 40 may reduce power consumption by not driving the display unit 12 and the communication unit 42.

[0023] In low power consumption mode, the second control unit 40 supplies power to the IMU 34, and the IMU 34 provides measurement results to the second control unit 40 at predetermined intervals. The second control unit 40 determines from the measurement results of the IMU 34 whether the electronic device 10 is stationary or not (S22). If it is not stationary (N in S22), it is possible that the user is rotating the wheels 14 of the electronic device 10, causing the power generation unit 30 to generate power, or that the user is moving the electronic device 10 to place it in one of the areas on the mat 2. In any case, if the electronic device 10 is not stationary, it is unlikely that the electronic device 10 is in a state where it can photograph the code attached to the mat 2, so the second control unit 40 maintains a state where it does not supply power to the image sensor 16, does not drive the image sensor 16, and avoids unnecessary power consumption (S24).

[0024] On the other hand, when the electronic device 10 is stationary (Y in S22), the electronic device 10 is positioned on the mat 2 and may be in a state where it can photograph the code attached to the mat 2, so the second control unit 40 supplies power to the image sensor 16 to drive it (S26). The image sensor 16 periodically photographs the area around the electronic device 10, and the second control unit 40 analyzes the images taken by the image sensor 16 to search whether the images contain the code attached to the mat 2 (S28). When the second control unit 40 reads a new code (Y in S28), it identifies the object information contained in the code and displays the object image corresponding to that object information on the display unit 12 (S30). Note that a newly read code means that the code read this time is different from the code read last time.

[0025] Figure 7 shows the state in which an object image is displayed on the display unit 12. The second control unit 40 identifies object information from the read code and displays the object image associated with that object information on the display unit 12. The electronic device 10 holds a database (not shown) that records object images corresponding to the object information, and the second control unit 40 reads the object image from the database and displays it on the display unit 12.

[0026] As described above, codes are assigned to each area partitioned on mat 2, and therefore the second control unit 40 displays the object image associated with the area in which the electronic device 10 is placed on the display unit 12 each time the electronic device 10 is placed in a different area. The display unit 12 may be electronic paper with image storage capabilities. Electronic paper does not consume power to continue displaying an image after it has been displayed once, thus contributing to power saving of the electronic device 10.

[0027] In S28, if the code read by the second control unit 40 is the same as the previous one (N in S28), the second control unit 40 does not update the display in the display unit 12. Therefore, if the electronic device 10 remains in the same area, the display unit 12 continues to display the same object image. If the stored energy is not zero (N in S32), the second control unit 40 repeatedly executes the process from S22, and when the stored energy becomes zero (Y in S32), this flow terminates.

[0028] The second control unit 40 may control the shooting cycle of the image sensor 16 according to the amount of charge stored in the power storage unit 32. When the amount of charge is small, the second control unit 40 may set the shooting cycle of the image sensor 16 to be relatively long, and when the amount of charge is large, it may set the shooting cycle of the image sensor 16 to be relatively short.

[0029] Furthermore, the second control unit 40 may control the display content of the display unit 12 according to the amount of charge stored in the power storage unit 32. When the amount of charge is low, the second control unit 40 may display content on the display unit 12 that consumes relatively less power, and when the amount of charge is high, it may display content on the display unit 12 that consumes relatively more power. Specifically, the second control unit 40 may display images with richer colors and animations when the amount of charge is high, and may also increase the frequency of image updates. For example, if the database associates multiple object images with one object piece of information, the second control unit 40 may determine the frequency of switching between object images according to the amount of charge.

[0030] Furthermore, it is preferable that the second control unit 40 erases the image on the display unit 12 when the amount of charge stored in the power storage unit 32 falls below a predetermined threshold. Since the display unit 12 in this embodiment is electronic paper, it will continue to display the object image even when the amount of charge becomes zero. Therefore, it is preferable that the second control unit 40 erases the display image on the display unit 12 just before the amount of charge becomes zero so that the object image does not continue to be displayed.

[0031] In the electronic device 10, the motor constituting the power generation unit 30 is driven by the power stored in the power storage unit 32 when power generation is not in progress, to rotate the wheels 14. The second control unit 40 has a function to control the rotation of the motor when power generation is not in progress, and may autonomously travel on the mat 2 based on destination information contained in the code. By using the motor for both power generation and travel in this way, the space of the housing 20 can be used effectively.

[0032] Figure 8 shows an example of an information processing system 1 comprising an electronic device 10 and an information processing device 50. The information processing device 50 is configured to have a camera that photographs the electronic device 10. The information processing device 50 is a mobile terminal device such as a smartphone or tablet, or smart glasses, and generates augmented reality (AR) images and displays the AR images on the display unit 58.

[0033] Figure 9 shows the functional blocks of the information processing device 50. The information processing device 50 includes a camera 52, an image recognition unit 54, a processing unit 56, and a display unit 58. The information processing device 50 includes a computer, and various functions are realized by the computer executing programs. The computer includes, as hardware, memory for loading programs, one or more processors for executing loaded programs, auxiliary storage devices, and other LSIs. The processor is composed of multiple electronic circuits, including semiconductor integrated circuits and LSIs, and these multiple electronic circuits may be mounted on one chip or on multiple chips. The functional blocks shown in Figure 9 are realized through the cooperation of hardware and software, and therefore, it will be understood by those skilled in the art that these functional blocks can be realized in various ways by hardware alone, software alone, or a combination thereof.

[0034] The processing unit 56 displays the camera image captured by the camera 52 in real space on the display unit 58. In the example shown in Figure 8, the camera 52 is capturing the electronic device 10, and the display unit 58 displays the real space including the electronic device 10. When the image recognition unit 54 recognizes an object image included in the camera image, it provides the processing unit 56 with the recognition result, that is, information identifying the recognized object image (object image ID). The processing unit 56 maintains a database that records 3D images corresponding to the object image ID, and reads the 3D image from the database based on the object image ID identified by the image recognition unit 54.

[0035] The processing unit 56 superimposes the read 3D image onto the camera image displayed on the display unit 58. As a result, the display unit 58 displays the 3D image corresponding to the result recognized by the image recognition unit 54 superimposed on the object image. In this way, the processing unit 56 superimposes the 3D image onto the camera image, allowing the user to enjoy augmented reality (AR).

[0036] The present invention has been described above based on examples. The examples are illustrative, and it will be understood by those skilled in the art that various modifications are possible in combinations of their components and processing processes, and that such modifications also fall within the scope of the present invention. The electronic device 10 may transmit information regarding the amount of charge from the communication unit 42 to the information processing device 50.

[0037] In this embodiment, the power generation unit 30 was described as a motor connected to the wheel 14, but other types of power generation modules may be used. For example, the power generation unit 30 may be a solar power generation module or a vibration power generation module. Also, in this embodiment, the display unit 12 was described as electronic paper, but other types of displays such as liquid crystal displays or organic EL displays may be used.

[0038] The information processing system 1 may contain multiple electronic devices 10. The electronic devices 10 communicate their charge levels to each other via the communication unit 42, and the electronic device 10 with the highest charge level may display information on the display unit 12 informing the user that it has a high charge level. By knowing the information regarding the charge levels, the user can get an opportunity to generate power. The electronic devices 10 may also have a contactless power supply function so that they can supply power from the electronic device 10 with a high charge level to the electronic device 10 with a low charge level.

[0039] In the embodiment, the electronic device 10 was described as a toy that a user can hold and play with, but the electronic device 10 does not have to be a toy, nor does it have to be small enough for a user to hold in their hand. The electronic device 10 may be a mobile device (device) having wheels 14, such as a wheeled robot, trolley, vehicle, walker, wheelchair, radio-controlled car, or robotic vacuum cleaner. Alternatively, the electronic device 10 may be a device (device) without wheels 14, such as a mobile terminal device like a smartphone or tablet, or furniture or electrical appliances.

[0040] In the embodiment, the second control unit 40 was described as identifying a code contained in the image captured by the image sensor 16 and displaying the image corresponding to that code on the display unit 12. However, the display unit 12 may also display an image corresponding to a subject other than a code. The image sensor 16 may be provided not only on the bottom surface of the housing 20, but also on the front, right side, back, left side, top surface, etc. of the housing 20 so as to capture the environment around the housing 20. The second control unit 40 analyzes the captured image to identify a subject and displays the image corresponding to that subject on the display unit 12. For example, the second control unit 40 may identify a subject such as an object, a person's face, a person's posture or movement, or an animal contained in the captured image and display the image corresponding to that subject on the display unit 12.

[0041] Furthermore, in the information processing system 1, the information processing device 50 may display information on the display unit 58 using AR (Augmented Reality) to increase the user's motivation for power generation, such as information to improve the efficiency of power generation and information regarding the actual amount of stored energy.

[0042] In the embodiment, it was explained that the electronic device 10 does not have a secondary battery or primary battery that is charged by an external power source. However, in a modified example, the electronic device 10 may have a secondary battery and / or primary battery that is charged by an external power source as an auxiliary power source. For example, the second control unit 40 may use the power from the power storage unit 32 as long as the amount of power stored in the power storage unit 32 is not depleted, and when the amount of power stored in the power storage unit 32 is depleted, it may use the auxiliary power source. [Industrial applicability]

[0043] This disclosure can be used in the technical field of electronic devices used by users. [Explanation of Symbols]

[0044] 1... Information processing system, 2... Mat, 10... Electronic equipment, 12... Display unit, 14, 14a, 14b... Wheels, 16... Image sensor, 18... Lighting unit, 20... Housing, 30... Power generation unit, 32... Energy storage unit, 34... IMU, 36... Control unit, 38... First control unit, 40... Second control unit, 42... Communication unit, 50... Information processing device, 52... Camera, 54... Image recognition unit, 56... Processing unit, 58... Display unit.

Claims

1. It is an electronic device, A power generation unit that generates electricity, A power storage unit that stores the electricity obtained from the power generation unit, An image sensor that captures images of the area around the electronic device, A display unit that displays an image corresponding to the subject captured by the image sensor, A control unit that controls the driving of the image sensor and the display unit, Equipped with an inertial measuring device, The image sensor and the display unit are driven by the power stored in the energy storage unit. The control unit, upon determining from the measurement results of the inertial measuring device that the electronic device is stationary, drives the image sensor; otherwise, it does not drive the image sensor. An electronic device characterized by the following features.

2. The display unit displays an image corresponding to a code contained in the image captured by the image sensor. The electronic device according to feature 1.

3. Equipped with multiple wheels, The power generation unit has a motor that generates electricity in conjunction with the rotation of the wheel. The electronic device according to feature 1.

4. The motor, when not generating power, is driven by the electricity stored in the power storage unit to rotate the wheel. The electronic device according to feature 3.

5. The control unit is A first control unit monitors the amount of energy stored in the energy storage unit, It includes a second control unit that controls the driving of the image sensor and the display unit, The first control unit supplies power to the second control unit when the amount of energy stored in the energy storage unit exceeds a predetermined value. The electronic device according to feature 1.

6. The second control unit controls the shooting cycle of the image sensor and / or the display content of the display unit according to the amount of charge stored in the power storage unit. The electronic device according to feature 5.

7. The aforementioned display unit is electronic paper. The electronic device according to feature 1.

8. An information processing system comprising an electronic device and an information processing device having a camera for photographing the electronic device, The aforementioned electronic device is A power generation unit that generates electricity, A power storage unit that stores the electricity obtained from the power generation unit, An image sensor that captures images of the area around the electronic device, A first display unit that displays an image corresponding to the subject captured by the image sensor, A control unit that controls the operation of the image sensor and the first display unit, Equipped with an inertial measuring device, The aforementioned information processing device is A recognition unit that recognizes the image captured by the camera, The system includes a second display unit that displays an image corresponding to the result recognized by the recognition unit, The image sensor and the first display unit are driven by the power stored in the power storage unit. The control unit, upon determining from the measurement results of the inertial measuring device that the electronic device is stationary, drives the image sensor; otherwise, it does not drive the image sensor. An information processing system characterized by the following:

9. The first display unit displays an image corresponding to a code contained in the image captured by the image sensor. The information processing system according to feature 8.

Images