Operation detection device and operation detection method

Abstract

An operation detection device in one embodiment is an operation detection device for detecting an object approaching an operation unit. The operation detection device includes: a sensor for detecting the distance to the object as a plurality of pixels and an object detection unit for detecting the object. The object detection unit determines a first pixel corresponding to a shortest distance having the shortest distance among a plurality of pixels, scans a plurality of second pixels located around the first pixel, and detects the object when the number of second pixels among the plurality of second pixels, in which the difference between the distance corresponding to each second pixel and the shortest distance is less than a fixed value, is greater than or equal to a predetermined number.

Description

Technical Field

[0001] The present disclosure relates to an operation detection device and an operation detection method. Background Art

[0002] Patent document 1 discloses a user interface device using a distance image sensor. The user interface device includes a transparent display that displays an image, a distance image sensor that acquires a distance image for measuring the distance to an object, and a distance image analysis unit that analyzes the distance image to detect a user's input operation. In the user interface device, the distance image sensor recognizes the center of gravity of an object at a distance close to a certain extent as a point position, and the distance image analysis unit detects the user's input operation based on the recognized point position.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2012-3585 Summary of the invention

[0006] Problem that the invention aims to solve

[0007] In addition, in the aforementioned operation detection device that acquires a distance image to detect an object (target), the sensor detects the distance to the target as a plurality of pixels. Each pixel acquired by the sensor corresponds to the distance between each point on the surface of the target and the sensor. The operation detection device determines any one of the plurality of pixels acquired by the sensor as a pixel indicating a close distance to the target, thereby detecting the target.

[0008] However, this type of operation detection device sometimes erroneously detects pixels indicating a close distance to an object even though the object does not exist, and the pixel sometimes becomes noise. That is, noise is included in the pixels detected by the sensor, thereby having the possibility of erroneously detecting an object even though the object does not exist. Therefore, there is room for improvement in the accuracy of object detection. In addition, in the case of calculating the center of gravity of an object as in the aforementioned user interface device, it is considered necessary to obtain the shape of the object. In this case, there will be a problem that the load of calculation processing is large and the detection of the object takes time.

[0009] An object of the present disclosure is to provide an operation detection device and an operation detection method capable of detecting an object with high accuracy and high speed.

[0010] Solutions for solving problems

[0011] An operation detection device in one embodiment is an operation detection device that detects an object approaching an operation unit. The operation detection device includes: a sensor that detects a distance to an object as a plurality of pixels; and an object detection unit that detects the object. The object detection unit determines a first pixel corresponding to a shortest distance in which the distance is the shortest among the plurality of pixels, scans a plurality of second pixels located around the first pixel, and detects the object when the number of second pixels in which the difference between the distance corresponding to each second pixel and the shortest distance is less than a fixed value among the plurality of second pixels is greater than or equal to a predetermined number.

[0012] An operation detection method in one embodiment is an operation detection method for detecting an object approaching an operation unit by using a sensor. The operation detection method includes the following steps: detecting the distance from the object to the sensor as a plurality of pixels; determining a first pixel corresponding to the shortest distance among the plurality of pixels; and scanning a plurality of second pixels located around the first pixel, and detecting the object when the number of second pixels among the plurality of second pixels, the number of which the difference between the distance corresponding to each second pixel and the shortest distance is less than a fixed value, is greater than or equal to a predetermined number.

[0013] In the aforementioned operation detection device and operation detection method, the sensor detects the distance to the object as a plurality of pixels, and the object detection unit determines the first pixel corresponding to the shortest distance at which the distance to the object is the shortest. The object detection unit scans a plurality of second pixels located around the determined first pixel, and detects the object when the number of second pixels corresponding to the distance whose difference with the shortest distance is less than a fixed value among the scanned plurality of second pixels is greater than a prescribed number. On the other hand, the object detection unit does not detect the object when the number of second pixels corresponding to the distance whose difference with the shortest distance is less than a fixed value among the scanned plurality of second pixels is less than a prescribed number. In this way, the object is not detected when the number of second pixels is less than a prescribed number, thereby determining that the determined first pixel is noise. As a result, the influence of noise can be eliminated, thereby suppressing the erroneous detection of the object, and thus the object can be detected with high precision. In addition, in the operation detection device and operation detection method, the processing of acquiring the shape of the object is not required when detecting the object, so the processing load can be reduced to increase the processing speed. Thus, the object can be detected at a high speed.

[0014] The operation detection device may further include a determination unit that determines whether the object detected by the object detection unit is an operation on the operation unit. The operation detection device may include the determination unit, thereby being able to determine whether the operation unit is operated.

[0015] The operating unit can be displayed as a virtual image. When the operating unit is displayed as a virtual image, the operating unit can be displayed in a floating manner, thereby improving the visibility of the operating unit. When the operating unit is displayed as a virtual image, it is assumed that sometimes the position of the operating unit recognized by the user deviates, so that the operating unit is operated inappropriately. Even in such a case, in the operation detection device and the operation detection method, it is possible to suppress the erroneous detection of the object. Therefore, according to the operation detection device and the operation detection method, the object can be detected with high accuracy.

[0016] Effects of the Invention

[0017] According to the present disclosure, it is possible to detect an object with high accuracy and high speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a structural diagram showing an operation detection device according to one embodiment.

[0019] Figure 2 Yes means through Figure 1 A conceptual diagram of a virtual image displayed on a display unit of an operation detection device.

[0020] Figure 3 FIG. 1 is a diagram showing an example of an operation unit displayed as a virtual image.

[0021] Figure 4 yes Figure 1 Functional block diagram of the control unit of the operation detection device.

[0022] Figure 5 It means by Figure 1 FIG. 1 is a diagram showing an example of distance image data output by a sensor of an operation detection device.

[0023] Figure 6 1 is a diagram showing an example of a first pixel of range image data and second pixels located around the first pixel.

[0024] Figure 7 Yes means Figure 1 A flowchart of an example of the operation of the operation detection device.

[0025] Figure 8 It is detailed Figure 7 A flowchart of a portion of a process. DETAILED DESCRIPTION

[0026] Hereinafter, embodiments of the operation detection device and the operation detection method according to the present disclosure will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same elements are denoted by the same reference numerals, and repeated descriptions are omitted.

[0027] Figure 1: is a structural diagram of the operation detection device 1 of the present embodiment. The operation detection device 1 is, for example, an operation detection device for constructing an HMI (Human Machine Interface) between a vehicle and a passenger (user) of the vehicle. The operation detection device 1 displays the operation unit 2 as a virtual image 3 in the air on the near front side when viewed from the user, and detects an object 4 approaching the operation unit 2. The operation unit 2 is, for example, a part that can activate various devices mounted on the vehicle (such as a vehicle-mounted rear camera or air conditioner, etc.), and as an example, it is a button such as a switch displayed as a virtual image 3. The object 4 operates the operation unit 2 to activate the device, and is, for example, a user's finger or a stick-like object such as a pen.

[0028] The operation detection device 1 detects the operation of the object 4 on the operating unit 2 based on the position of the detected object 4, and operates each device based on the detected operation. As examples of the operation, a pressing operation, a touch operation, or a sliding operation performed by the object 4 on the operating unit 2 is cited, but the type of operation is not particularly limited. In this embodiment, a pressing operation is exemplified as an operation of the operating unit 2. The pressing operation includes both an operation of pressing the operating unit 2 downward and an operation of pressing the operating unit 2 upward.

[0029] like Figure 1 As shown, the operation detection device 1 includes a display unit 10, a depth sensor 20 (sensor), and a control unit 30. The display unit 10 displays the operation unit 2 as a virtual image 3 in the air. The display unit 10 includes an AI (Aerial Imaging) board 11 (registered trademark) as an aerial imaging element and a liquid crystal panel 12. The AI ​​board 11 is made using the technology described in Japanese Patent No. 4865088, for example.

[0030] The liquid crystal panel 12 is, for example, a display of a mobile terminal such as a PC (personal computer), a tablet terminal, or a mobile phone. The liquid crystal panel 12 is, for example, arranged in such a way that its display surface is substantially horizontal. The AI ​​board 11 is inclined obliquely relative to the liquid crystal panel 12 (horizontally). The inclination angle of the AI ​​board 11 relative to the liquid crystal panel 12 can be set to be variable. The liquid crystal panel 12 displays an image based on a signal output from the control unit 30.

[0031] Figure 2 3 is a conceptual diagram showing a virtual image 3 displayed by the display unit 10. Figure 2 As shown, the image displayed by the liquid crystal panel 12 is displayed as a virtual image 3 through the AI ​​board 11 at a position closer to the user U relative to the AI ​​board 11 and the liquid crystal panel 12. For example, light L1 emitted upward from the liquid crystal panel 12 and then incident on the AI ​​board 11 is reflected twice by the AI ​​board 11, and is imaged as a virtual image 3 in a space closer to the front side than the display unit 10 when viewed from the user U.

[0032] Figure 3 FIG. 2 is an example of the operation unit 2 displayed as a virtual image 3. Figure 3 As shown, the operation unit 2 is, for example, an operation screen capable of operating the vehicle-mounted rear camera, and the photographing position of the vehicle-mounted rear camera can be switched by pressing each button of the operation unit 2. As an example, the operation unit 2 includes a first virtual image button 2A for photographing the left side of the vehicle, a second virtual image button 2B for photographing the rear side of the vehicle, and a third virtual image button 2C for photographing the right side of the vehicle.

[0033] Refer again Figure 1 . The depth sensor 20 is arranged on the opposite side of the object 4 in a manner of sandwiching the operation unit 2. In one example, the depth sensor 20 is arranged on a virtual straight line connecting the operation unit 2 and the object 4. The depth sensor 20 acquires distance image data, which includes information on the position (two-dimensional position) of the object 4 on a plane perpendicular to the virtual straight line and information on the distance D1 from the depth sensor 20 to the object 4. The distance image data is acquired, for example, in 640×480 pixels. The depth sensor 20 outputs the acquired distance image data to the control unit 30 at a predetermined period (for example, 1 / 30 second).

[0034] Specifically, the depth sensor 20 irradiates light (e.g., infrared light) to each point on an object existing in a photographic area including the object 4, and receives light reflected from each point on the object. The depth sensor 20 measures the distance between the depth sensor 20 and each point on the object based on the received light, and outputs the measured distance per pixel.

[0035] The distance between the depth sensor 20 and each point on the object is measured, for example, by a Light Coding method. In this method, the depth sensor 20 irradiates light with a random dot pattern to each point on the object existing in the photographic area including the object 4. In addition, the depth sensor 20 receives the light reflected from each point on the object, detects the deformation of the pattern of the reflected light, and thereby measures the distance between the depth sensor 20 and each point on the object. The depth sensor 20 detects the information of the two-dimensional position of each point on the object and the information of the distance from the depth sensor 20 to each point on the object as a plurality of pixels, and outputs the detected plurality of pixels to the control unit 30.

[0036] The control unit 30 can communicate with the depth sensor 20 and the liquid crystal panel 12. The control unit 30 includes, for example, a CPU 31 (Central Processing Unit) that executes a program, a storage unit 32 such as a ROM (Read Only Memory) and a RAM (Random Access Memory), an input / output unit 33, and a driver 34. The input / output unit 33 is operated under the control of the CPU 31 to read and write data in the storage unit 32, thereby realizing the functions realized by each functional component of the control unit 30. The form and arrangement location of the control unit 30 are not particularly limited.

[0037] Figure 4 30 is a functional block diagram of the control unit 30. Figure 4 As shown, the control unit 30 includes an image output unit 40, an object detection unit 41, an operation determination unit 42, and a signal output unit 43 as functional components. The image output unit 40 outputs image data of an image displayed on the liquid crystal panel 12 to the liquid crystal panel 12. The image displayed on the liquid crystal panel 12 is not limited to Figure 3 The liquid crystal panel 12 can display various images based on the image data from the image output unit 40 .

[0038] The object detection unit 41 detects the object 4 by identifying a pixel indicating that the distance between the depth sensor 20 and the object 4 is short from among a plurality of pixels included in the distance image data output from the depth sensor 20 . Figure 5 2 is a diagram schematically showing the distance image data D output from the depth sensor 20. When the distance image data D from the depth sensor 20 is input, the object detection unit 41 sets a scanning area R1 for the distance image data D.

[0039] The scanning area R1 is, for example, an area to be scanned in the distance image data D. The object detection unit 41 sets, for example, an area including the center of the distance image data D as the scanning area R1. In this way, the object detection unit 41 sets the area including the center of the distance image data D (that is, the area excluding the ends of the distance image data D) as the scanning area R1, thereby increasing the speed of detecting the object 4. The object detection unit 41 may also set the entire distance image data D as the scanning area R1. The object detection unit 41 sets a temporary reference value T1 as a value representing the shortest shortest distance between the depth sensor 20 and each point on the object 4. The temporary reference value T1 is, for example, 300 mm, but can be appropriately changed.

[0040] For example, the object detection unit 41 sets the position of a pixel EA at a corner on the diagonal of the scanning area R1 set as a rectangular shape among the multiple pixels E in the scanning area R1 as the scanning start position, sets the position of a pixel EB at the other corner as the scanning end position, and scans each pixel E in the scanning area R1 from pixel EA to pixel EB in sequence. For each pixel E, there is information corresponding to the distance between the depth sensor 20 and each point on the object. The object detection unit 41 determines for each pixel E whether the distance corresponding to the pixel E is smaller than the temporary reference value T1. When the object detection unit 41 determines that the distance is smaller than the temporary reference value T1, the pixel E corresponding to the distance is determined as the first pixel. The first pixel is the pixel E corresponding to the shortest distance between each point on the object 4 and the depth sensor 20 among the multiple pixels E in the scanning area R1.

[0041] exist Figure 6 In the example shown, pixels E at which the distance between the object 4 and the depth sensor 20 is close are represented by dark colors, and pixels E at which the distance between the object 4 and the depth sensor 20 is far are represented by light colors. When the object detection unit 41 determines that the pixel E is the first pixel E1 corresponding to a distance smaller than the temporary reference value T1, it sets a surrounding area R2 centered on the first pixel E1 and scans a plurality of second pixels E2 within the surrounding area R2. The surrounding area R2 is, for example, an area having a size (range) corresponding to the size of the object 4. In the present embodiment, the surrounding area R2 is, for example, a rectangular area including 7×7 pixels centered on the first pixel E1. The second pixels E2 are, for example, all the pixels E within the surrounding area R2 except the first pixel E1. Figure 6 In the example shown, the second pixels E2 are 48 pixels E located around the first pixel E1.

[0042] The object detection unit 41 determines whether the number of second pixels E21 in the surrounding area R2, in which the difference between the distance corresponding to each second pixel E2 and the distance corresponding to the first pixel E1 is less than a fixed value (e.g., 10 mm), is greater than or equal to a predetermined number. The "predetermined number" is, for example, half of the total number of second pixels E2 or 70% of the total number of second pixels E2, but can be changed appropriately.

[0043] In this embodiment, the object detection unit 41 determines whether the number of second pixels E21 whose distances corresponding to the second pixels E2 and the distances corresponding to the first pixels E1 differ by 10 mm or less is 35 or more out of the total number of 48 second pixels E2. Figure 6 In the example shown, the second pixel E21 is represented by a lighter color than the first pixel E1, and the second pixel E22 whose distance difference is not less than 10 mm is represented by an even lighter color than the second pixel E21.

[0044] When the object detection unit 41 determines that the number of second pixels E21 is 35 or more, it updates the temporary reference value T1 using the distance corresponding to the first pixel E1 and records the two-dimensional position of the object 4. On the other hand, when the object detection unit 41 determines that the number of second pixels E21 is not 35 or more, it determines that the first pixel E1 is noise and does not update the temporary reference value T1.

[0045] After scanning all the pixels E in the scanning area R1, the object detection unit 41 determines the first pixel E1 corresponding to the shortest distance recorded as the temporary reference value T1 as a pixel indicating that the distance between the depth sensor 20 and the object 4 is short. The object detection unit 41 determines the first pixel E1 as a pixel indicating that the distance between the depth sensor 20 and the object 4 is short, thereby detecting the object 4. When detecting the object 4, the object detection unit 41 outputs position data indicating the position of the object 4 to the operation determination unit 42.

[0046] Refer again Figure 4 The operation determination unit 42 determines whether the operation unit 2 is pressed by the object 4 based on the position data output from the object detection unit 41. Figure 1 As shown, the operation determination unit 42 determines whether the distance D1 between the depth sensor 20 and the object 4, that is, the shortest distance corresponding to the first pixel E1 determined by the object detection unit 41, is less than a predetermined threshold value T2. The threshold value T2 is, for example, 100 mm, but can be changed appropriately. In addition, when the operation determination unit 42 determines that the distance D1 is less than the threshold value T2, it determines that the object 4 has reached the virtual pressing determination surface S and the pressing operation of the operation unit 2 has been performed.

[0047] The press determination surface S is a virtual surface formed at a location at a fixed distance from the depth sensor 20, and is provided at a position close to the operating unit 2. The position of the press determination surface S may coincide with the position of the operating unit 2, or may be a position separated from the operating unit 2 by a predetermined distance. In the present embodiment, the position of the press determination surface S coincides with the position of the operating unit 2.

[0048] When the operation determination unit 42 determines that the operation unit 2 is pressed, the signal output unit 43 generates a control signal based on the pressing operation of the operation unit 2. The signal output unit 43 outputs the generated control signal to the aforementioned vehicle-mounted rear camera and other equipment, and the equipment receives the control signal from the signal output unit 43 and operates. Figure 3In the example shown, when the operation determination unit 42 determines that the first virtual image button 2A, the second virtual image button 2B, or the third virtual image button 2C is pressed, the signal output unit 43 outputs a control signal to the vehicle-mounted rear camera. The vehicle-mounted rear camera displays any one of an image of the left side of the vehicle, an image of the rear of the vehicle, and an image of the right side of the vehicle based on the input control signal.

[0049] Next, the operation of the operation detection device 1 will be described. The operation of the operation detection device 1 is performed by the CPU 31 reading out and executing a program stored in the storage unit 32, for example. Figure 6 1 is a flowchart showing an example of the operation of the operation detection device 1. First, the depth sensor 20 acquires the distance image data D and outputs the acquired distance image data D to the control unit 30. The distance image data D includes information on the two-dimensional position of the object 4 and information on the distance D1 from the depth sensor 20 to the object 4. The object detection unit 41 detects the object 4 based on the input distance image data D (process P1). The details of process P1 will be described later. When the object detection unit 41 detects the object 4, it outputs position data indicating the position of the object 4 to the operation determination unit 42.

[0050] Next, the operation determination unit 42 determines whether the pressing operation of the operation unit 2 has been performed based on the position data output from the object detection unit 41 (step P2). Figure 1 As shown in FIG. 2 , it is determined whether the distance D1 between the depth sensor 20 and the object 4 is less than or equal to the threshold value T2. When the operation determination unit 42 determines that the distance D1 is less than or equal to the threshold value T2 (“Yes” in step P2), it is determined that the object 4 has reached the virtual pressing determination surface S and the pressing operation of the operation unit 2 has been performed. On the other hand, when the operation determination unit 42 determines that the distance D1 is not less than or equal to the threshold value T2 (“No” in step P2), it is determined that the object 4 has not reached the pressing determination surface S and the pressing operation of the operation unit 2 has not been performed. When it is determined that the pressing operation of the operation unit 2 has not been performed, the process returns to step P1 and the step P1 of detecting the object 4 is continued.

[0051] When the operation determination unit 42 determines that the operation unit 2 is pressed ("Yes" in step P2), the signal output unit 43 generates a control signal based on the pressing operation of the operation unit 2. The signal output unit 43 outputs the generated control signal to the aforementioned vehicle-mounted rear camera and other equipment, and the equipment receives the control signal from the signal output unit 43 and operates (step P3). Then, a series of steps are completed.

[0052] Here, the step P1 in which the object detection unit 41 detects the object 4 will be described in detail. Furthermore, the operation detection method of this embodiment will also be described. Figure 8 It is detailed Figure 7 First, the depth sensor 20 detects the distance image data D including the information of the two-dimensional position of each point on the object and the information of the distance from the depth sensor 20 to each point on the object as a plurality of pixels E, and outputs the detected plurality of pixels E to the control unit 30 (step P11).

[0053] Next, the object detection unit 41 sets the scanning area R1 for the distance image data D input from the depth sensor 20 (step P12). For example, the object detection unit 41 sets the position of the pixel EA located at one corner on the diagonal line of the scanning area R1 set to a rectangular shape as the scanning start position. Thereafter, the object detection unit 41 sets a temporary reference value T1 as a value indicating the shortest distance between the depth sensor 20 and each point on the object 4 (step P13).

[0054] Next, the object detection unit 41 scans each pixel E in the scanning area R1 in sequence, starting from the pixel EA, and determines whether the distance corresponding to the pixel E is smaller than the temporary reference value T1 (process P14). When the object detection unit 41 determines that the distance is smaller than the temporary reference value T1 ("Yes" in process P14), the pixel E corresponding to the distance is determined as the first pixel E1 (process P15). On the other hand, when the object detection unit 41 determines that the distance corresponding to the pixel E is not smaller than the temporary reference value T1 ("No" in process P14), the pixel E corresponding to the distance is not determined as the first pixel E1, and the next pixel E in the scanning area R1 is scanned (process P19).

[0055] After determining that the distance corresponding to the pixel E is smaller than the temporary reference value T1 and determining the first pixel E1, the object detection unit 41 sets the surrounding area R2 with the determined first pixel E1 as the center, and scans the plurality of second pixels E2 in the surrounding area R2 (process P16). Thereafter, the object detection unit 41 determines whether the number of second pixels E21 in the plurality of second pixels E2 in the surrounding area R2, in which the difference between the distance corresponding to each second pixel E2 and the distance corresponding to the first pixel E1 is less than a fixed value, is greater than or equal to a prescribed number (process P17). In the present embodiment, the object detection unit 41 determines whether the number of second pixels E21 in which the difference between the distance corresponding to each second pixel E2 and the distance corresponding to the first pixel E1 is less than 10 mm, is greater than or equal to 35.

[0056] When the object detection unit 41 determines that the number of second pixels E21 is 35 or more ("Yes" in step P17), it updates the temporary reference value T1 with the distance corresponding to the first pixel E1 (step P18). At this time, the object detection unit 41 records the two-dimensional position of the object 4. Thereafter, the object detection unit 41 scans the next pixel E in the scanning area R1 (step P19). On the other hand, when the object detection unit 41 determines that the number of second pixels E21 is not 35 or more ("No" in step P17), it determines that the first pixel E1 is noise and does not update the temporary reference value T1. Then, step P19 is executed.

[0057] After scanning the next pixel E in the scanning area R1, the object detection unit 41 determines whether the pixel E is the pixel EB corresponding to the scanning end position (process P20). When the object detection unit 41 determines that the pixel E is the pixel EB ("Yes" in process P20), the first pixel E1 corresponding to the shortest distance recorded as the temporary reference value T1 is determined as a pixel indicating that the distance between the depth sensor 20 and the object 4 is close, thereby detecting the object 4 (process P21). When the object detection unit 41 detects the object 4, the position data including the information of the position of the object 4 is output to the operation determination unit 42, and then the process P2 is executed. On the other hand, when the object detection unit 41 determines that the pixel E is not the pixel EB ("No" in process P20), it returns to process P14.

[0058] Next, the effects obtained by the operation detection device 1 and the operation detection method of this embodiment are described. Figure 6As shown, the depth sensor 20 detects the distance to the object 4 as a plurality of pixels E, and the object detection unit 41 determines the first pixel E1 corresponding to the shortest distance to the object 4. Furthermore, the object detection unit 41 scans the plurality of second pixels E2 located around the determined first pixel E1, and detects the object 4 when the number of second pixels E21 corresponding to the distance whose difference from the shortest distance is less than a fixed value among the scanned plurality of second pixels E2 is greater than a prescribed number. On the other hand, the object detection unit 41 does not detect the object 4 when the number of second pixels E21 corresponding to the distance whose difference from the shortest distance is less than a fixed value among the scanned plurality of second pixels E2 is less than a prescribed number. In this way, the object 4 is not detected when the number of second pixels E21 is less than a prescribed number, and thus the determined first pixel E1 can be determined as noise. As a result, the influence of noise can be eliminated, and the erroneous detection of the object 4 can be suppressed, so that the object 4 can be detected with high accuracy. Furthermore, in the operation detection device 1 and the operation detection method, since the process of acquiring the shape of the object 4 is not required when detecting the object 4, the processing load can be reduced and the processing speed can be increased. Thus, the object 4 can be detected at a high speed.

[0059] The object detection unit 41 detects the object 4 when the number of second pixels E21 corresponding to the distance whose difference from the shortest distance is less than a fixed value among the plurality of second pixels E2 in the surrounding area R2 is more than half or more than 70% of the total number of second pixels E2. Thus, it is possible to more accurately suppress the erroneous detection of the object 4 due to noise, and to detect the object 4 with higher accuracy. Furthermore, the surrounding area R2 set by the object detection unit 41 has a size corresponding to the size of the object 4. Thus, for example, it is possible to suppress the erroneous detection of non-objects smaller than the object 4, and to detect the object 4 with higher accuracy.

[0060] The operation detection device 1 includes the operation determination unit 42. This makes it possible to determine whether or not the operation unit 2 is pressed.

[0061] The operation unit 2 is displayed as a virtual image 3. When the operation unit 2 is displayed as the virtual image 3, the operation unit 2 can be displayed in a floating manner, so the visibility of the operation unit 2 can be improved. When the operation unit 2 is displayed as the virtual image 3, it is assumed that the position of the operation unit 2 recognized by the user U sometimes deviates, and the operation unit 2 is pressed inappropriately. Even in such a case, in the operation detection device 1 and the operation detection method of the present embodiment, it is possible to suppress the erroneous detection of the object 4. Therefore, according to the operation detection device 1 and the operation detection method of the present embodiment, the object 4 can be detected with high accuracy.

[0062] The above is a description of the embodiments of the operation detection device and the operation detection method involved in the present disclosure. However, the present disclosure is not limited to the aforementioned embodiments, and can be modified or applied to other methods without changing the scope of the subject matter recorded in the claims. That is, the content and sequence of each process of the operation detection method and the structure of each part of the operation detection device can be appropriately changed without changing the subject matter of the claims.

[0063] For example, in the aforementioned embodiment, the depth sensor 20 measures the distance between the depth sensor 20 and each point on the object by the Light Coding method, but is not limited to this method. For example, the depth sensor 20 can measure the distance between the depth sensor 20 and each point on the object by the TOF (Time of Flight) method. In the TOF method, the depth sensor 20 calculates the flight time (i.e., delay time) of the light from the light reflected at each point on the object to the depth sensor 20, and measures the distance between the depth sensor 20 and each point on the object based on the calculated flight time and the speed of light. Even in such a method, the same effect as the aforementioned embodiment is achieved.

[0064] In the aforementioned embodiment, the object detection unit 41 determines whether the number of second pixels E21 among the total number of second pixels E2, i.e., 48, is 35 or more. However, the total number of second pixels E2 is not limited to 48 and can be changed appropriately. The number of second pixels E21 as a criterion for determination is not limited to 35 and can be changed appropriately. The shape and display mode of each pixel E are not particularly limited and can be changed appropriately. In the aforementioned embodiment, the rectangular scanning area R1 and the surrounding area R2 are described, but the shape and display mode of these areas are not particularly limited.

[0065] In the above-mentioned embodiment, an example is described in which the operation unit 2 includes the first virtual button 2A, the second virtual button 2B, and the third virtual button 2C, but the layout and type of the operation unit can be changed appropriately. The operation detection device can be a device that detects an operation that causes various devices other than a vehicle to operate, and the operation detection device and operation detection method can also be applied to various devices other than a vehicle.

[0066] Description of Reference Numerals

[0067] 1: operation detection device; 2: operation unit; 3: virtual image; 4: object; 10: display unit; 20: depth sensor; 30: control unit; 41: object detection unit; 42: operation determination unit; E: pixel; E1: first pixel; E2, E21, E22: second pixels.

Claims

1. An operation detection device for detecting an object approaching an operation portion, the operation detection device comprising: a sensor that detects a distance to the object as a plurality of pixels; and an object detection unit that detects the object, in, The object detection unit determines a first pixel corresponding to the shortest distance among the plurality of pixels, scans a plurality of second pixels located around the first pixel, and detects the object when the number of second pixels among the plurality of second pixels, the number of which the difference between the distance corresponding to each second pixel and the shortest distance is less than a fixed value, is greater than or equal to a predetermined number. The sensor outputs distance image data, The object detection unit sets a scanning area based on the range image data. The object detection unit sets a temporary reference value as a value indicating the shortest distance between the sensor and each point on the object, The object detection unit sets the position of a third pixel at one corner on a diagonal line of the scanning area set as a rectangular shape among the plurality of pixels in the scanning area as a scanning start position, sets the position of a fourth pixel at another corner as a scanning end position, and sequentially scans each of the pixels in the scanning area from the third pixel to the fourth pixel. The object detection unit determines, for each of the pixels, whether the distance corresponding to the pixel is smaller than the temporary reference value. When the object detection unit determines that the distance is smaller than the temporary reference value, it specifies the pixel corresponding to the distance as the first pixel.

2. The operation detection device according to claim 1, wherein: The device further includes a determination unit that determines whether the object detected by the object detection unit has operated the operation unit.

3. The operation detection device according to claim 1 or 2, characterized in that: The operation portion is displayed as a virtual image.

4. An operation detection method, wherein a sensor is used to detect an object approaching an operation portion, the operation detection method comprising the following steps: detecting a distance from the object to the sensor as a plurality of pixels; determining a first pixel among the plurality of pixels corresponding to a shortest distance for which the distance is the shortest; as well as scanning a plurality of second pixels located around the first pixel, and detecting the object when the number of second pixels whose distances corresponding to the second pixels differ from the shortest distance by a fixed value or less among the plurality of second pixels is equal to or greater than a predetermined number, The process of inspecting the object is performed by an object detection unit. The sensor outputs distance image data, The object detection unit sets a scanning area based on the range image data. The object detection unit sets a temporary reference value as a value indicating the shortest distance between the sensor and each point on the object, The object detection unit sets the position of a third pixel at one corner on a diagonal line of the scanning area set as a rectangular shape among the plurality of pixels in the scanning area as a scanning start position, sets the position of a fourth pixel at another corner as a scanning end position, and sequentially scans each of the pixels in the scanning area from the third pixel to the fourth pixel. The object detection unit determines, for each of the pixels, whether the distance corresponding to the pixel is smaller than the temporary reference value. When the object detection unit determines that the distance is smaller than the temporary reference value, it specifies the pixel corresponding to the distance as the first pixel.

Images