Detection system

Abstract

To provide a motion detection system capable of mitigating impact due to an environment at use time and detecting the occurrence of a prescribed event with high accuracy.SOLUTION: A detection system in an embodiment of the present invention comprises: an acquisition unit for acquiring the sensor data outputted from a sensor; a generation unit for inputting sensor data corresponding to a first time of day by executing learning processing on a neural network using the sensor data acquired during a prescribed preparation period and generating, for each preparation time, a model that outputs a predictive value of sensor data corresponding to a second time of day after a prescribed time from the first time of day; and a determination unit for determining whether a prescribed event has occurred, on the basis of a difference between the predictive value outputted from the model and the actual sensor data corresponding to the second time of day.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a sensing system. [Background technology]

[0002] In recent years, trained models generated by machine learning are used to detect specific events in various technical fields. For example, when detecting the behavior of a vehicle or its occupants based on the detection results (sensor data) of a sensor mounted on the vehicle, the prediction results of a trained model may be used. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-180701 A [Patent Document 2] JP 2006-70704 A [Patent Document 3] JP 2019-106059 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, depending on the type of sensor and its installation location, the environment (e.g., weather) can have a large effect on the sensor data, and the sensor data may contain noise that was not anticipated during machine learning. In such cases, it may not be possible to achieve highly accurate detection even if a trained model is used.

[0005] Therefore, one of the problems that the embodiments of the present invention aim to solve is to provide a motion detection system that reduces the influence of the environment during use and is capable of detecting the occurrence of a predetermined event with high accuracy. [Means for solving the problem]

[0006] A detection system according to one embodiment of the present invention is characterized by comprising: an acquisition unit that acquires sensor data output from a sensor; a generation unit that generates a model for each preparation period by executing a learning process on a neural network using the sensor data acquired during a predetermined preparation period, the model inputting sensor data corresponding to a first time and outputting a predicted value of sensor data corresponding to a second time that is a predetermined time after the first time; and a determination unit that determines whether or not a predetermined event has occurred based on the difference between the predicted value output from the model and the actual sensor data corresponding to the second time.

[0007] According to the above configuration, a model adapted to the environment (e.g., weather, etc.) during the preparation period can be generated for each preparation period. Therefore, the determination process for determining whether or not a predetermined event has occurred can be executed using a model adapted to an environment close to the environment when the determination process is executed. This reduces the influence of the environment and enables the predetermined event to be detected with high accuracy even when the sensor is installed in a place where the environment changes greatly, such as outdoors.

[0008] In the above configuration, the neural network may include a reservoir layer and an output layer that outputs a predicted value by weighting an output from the reservoir layer with a predetermined output connection weight, and the learning process may include reservoir computing that updates the output connection weight based on an error between the predicted value and a ground truth value.

[0009] According to the above configuration, a model is generated by reservoir computing, which is capable of high-speed learning. This allows a preparation period to be set immediately before the execution of a judgment process, and allows the generation of a model that is adapted to an environment that is extremely close to the environment when the judgment process is executed.

[0010] In addition, in the above configuration, the sensor data may change according to the movements of a person within the detection range of the sensor, the preparation period may be a period during which no person is present within the detection range, and the event may be a predetermined movement performed by a person within the detection range.

[0011] According to the above configuration, a predetermined movement of a person can be detected with high accuracy by using sensor data from a sensor installed in a place where the environment changes greatly, such as outdoors.

[0012] In the above configuration, the sensor may be a capacitance sensor installed at a position corresponding to the opening and closing mechanism of the moving body, and the movement may be a kicking movement in which a person moves their foot in and out of the space below the moving body.

[0013] According to the above configuration, it is possible to improve the robustness of a system that automatically opens and closes an opening / closing mechanism (for example, a back door) of a moving body such as a vehicle by a kick action by a specific person.

[0014] In addition, in the above configuration, the preparation period may include a period from when a person carrying a key for operating the moving body enters an area within a first distance from the moving body to when the person reaches a position a second distance away from the moving body that is smaller than the first distance.

[0015] According to the above configuration, a model is generated while the person is approaching the moving object, and the kicking motion can be detected with high accuracy using a model that is adapted to an environment that is extremely similar to the environment in which the person performs the kicking motion. [Brief description of the drawings]

[0016] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of a vehicle according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of the configuration of the door opening / closing system and the detection system according to the embodiment. [Diagram 3] FIG. 3 is a diagram illustrating an example of a functional configuration of the determination ECU according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of a model and a learning process according to the embodiment. [Diagram 5] FIG. 5 is a diagram illustrating an example of a preparation period according to the embodiment. [Figure 6] FIG. 6 is a flowchart illustrating an example of a process for generating a model in the detection system of the embodiment. [Figure 7] FIG. 7 is a diagram showing an example of a method for determining a kick motion in the embodiment. [Figure 8] FIG. 8 is a flowchart showing an example of a process for determining the presence or absence of a kicking motion in the detection system of the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Exemplary embodiments of the present invention are disclosed below. The configurations of the embodiments described below, and the actions, results, and effects brought about by the configurations are merely examples. The present invention can be realized by configurations other than those disclosed in the following embodiments, and it is possible to obtain at least one of various effects and derivative effects based on the basic configuration.

[0018] 1 is a diagram showing an example of the configuration of a vehicle 1 according to an embodiment. The vehicle 1 (an example of a moving body) according to the present embodiment is equipped with a door opening / closing system S that automatically opens and closes a back door 3 (an example of an opening / closing mechanism) provided at the rear of a vehicle body 2 in response to a kicking motion (an example of a predetermined motion) of a specific person (e.g., a person carrying a key for driving the vehicle 1). The kicking motion is a motion in which a specific person moves a foot F in and out of a specific portion of the vehicle body 2 (in this example, the space below the back door 3).

[0019] The door opening / closing system S includes a detection system S1 that detects that a specific person has performed a kicking motion. The detection system S1 includes a kick sensor 11 (an example of a sensor) installed at the bottom of the back door 3 (behind the rear bumper 4). The kick sensor 11 of this embodiment outputs sensor data that changes according to a kicking motion performed within a detection range (such as the area below the rear bumper 4). The kick sensor 11 may be, for example, a capacitance sensor. In this case, the kick sensor 11 outputs sensor data indicating a capacitance value that changes according to the displacement of an object such as a foot F within the detection range.

[0020] The above configuration is an example, and the configurations of the door opening / closing system S, the detection system S1, and the mechanism (mobile body such as the vehicle 1) in which they are mounted are not limited to those described above. For example, a case where the opening / closing mechanism that is automatically opened and closed by a kick operation is only the back door 3 will be described here as an example, but other opening / closing mechanisms such as the sliding door 5 may also be automatically opened and closed. In this case, a kick sensor 11 is also mounted on a portion corresponding to the sliding door 5 (for example, the lower part of the sliding door 5, etc.).

[0021] 2 is a diagram showing an example of the configuration of a door opening / closing system S and a detection system S1 according to the embodiment. The door opening / closing system S includes a detection system S1, a door control ECU (Electronic Control Unit) 21, a lock mechanism 22, and a drive mechanism .

[0022] The detection system S1 includes a kick sensor 11 and a determination ECU 12. The determination ECU 12 executes a process for determining whether or not a kick motion has been performed by a specific person based on the sensor data output from the kick sensor 11. A specific functional configuration of the determination ECU 12 will be described later. The determination ECU 12 is an information processing device configured using appropriate hardware and software, and can be configured using, for example, a CPU (Central Processing Unit), a memory, an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), etc.

[0023] The lock mechanism 22 is a mechanism that switches the opening and closing mechanism of the back door 3 etc. between a locked state and an unlocked state, and may be configured using an appropriate locking structure, actuator, etc. The drive mechanism 23 is a mechanism that automatically opens and closes the back door 3, and may be configured using an actuator that rotates the back door 3 around a rotation axis, etc.

[0024] The door control ECU 21 executes processing for controlling the lock mechanism 22 and the drive mechanism 23 based on the determination result by the determination ECU 12. Specifically, when a determination result indicating that a kick action has been performed by a specific person is output from the determination ECU 12, the door control ECU 21 controls the lock mechanism 22 and the drive mechanism 23 to automatically open the closed back door 3 or automatically close the open back door 3. The door control ECU 21 is an information processing device configured using appropriate hardware and software, and can be configured using, for example, a CPU, a memory, an FPGA, an ASIC, etc.

[0025] 3 is a diagram showing an example of a functional configuration of the determination ECU 12 of the embodiment. The determination ECU 12 of the present embodiment includes an acquisition unit 101, a generation unit 102, and a determination unit 103. These functional units 101 to 103 may be configured by cooperation between hardware elements and software elements (programs, etc.) that configure the determination ECU 12. Furthermore, at least one of these functional units 101 to 103 may be configured by dedicated hardware (circuits, etc.).

[0026] The acquisition unit 101 acquires the sensor data output from the kick sensor 11 .

[0027] The generation unit 102 generates a model M, which is a trained model that can be used to determine the presence or absence of a kicking motion, by a learning process for a neural network (an integrated circuit that imitates a neuron network structure, a mathematical model, etc.). The generation unit 102 of this embodiment has a learning processing unit 111 that generates a model M for each preparation period by executing a learning process for the neural network using sensor data acquired during a predetermined preparation period. The model M inputs sensor data corresponding to a first time and outputs a predicted value of the sensor data corresponding to a second time that is a predetermined time after the first time. The model M and the learning process will be described later.

[0028] The judgment unit 103 judges whether or not a kicking action has been performed by a specific person based on the difference between the predicted value output from the model M and the actual sensor data corresponding to the second time, and outputs the judgment result to the door control ECU 21.

[0029] Fig. 4 is a diagram showing an example of a model M and a learning process according to an embodiment. Fig. 4 shows an example of a process in a learning stage for generating the model M. The model M according to this embodiment is configured using a neural network including an input layer 201, a reservoir layer 202, and an output layer 203, and is generated using a technique called reservoir computing.

[0030] The input layer 201 receives the sensor data value u(t) output from the kick sensor 11 as an input value, and assigns a predetermined input connection weight W in Here, t indicates a detection time in a series of time series (for example, the time when sensor data is acquired), and input values ​​u(t), u(t+1), u(t+2), ... for each detection time t, t+1, t+2, ... are input to the input layer 201.

[0031] The reservoir layer 202 is composed of a recurrent network in which multiple neurons are randomly connected, and each neuron has a state value x(t). The state value x(t) is calculated by weighting the past state value x(t-1) with a predetermined recurrent connection weight W, the value output from the input layer 201, and the value y(t) output from the output layer 203 with a predetermined feedback connection weight W. fb It is determined by the weighted value of and.

[0032] The output layer 203 assigns a predetermined output connection weight W outThe weighted value y(t) is output as an output value. The output value y(t) is a predicted value of the input value u(t+1) of the sensor data corresponding to the second time t+1 when the input value u(t) of the sensor data corresponding to the first time t is input. The second time t+1 is a time when a predetermined time has elapsed since the first time t. The first time t and the second time t+1 are times during a preparation period described later. The past time t-1 is a time immediately before the first time t.

[0033] In the learning process of this embodiment, the input connection weights W in , the recurrent connection weights W and the feedback weights W fb is fixed, and the output connection weights W out Only the output connection weight W is updated based on the error between the predicted value y(t) and the correct value (the input value of the sensor data corresponding to the second time t+1) u(t+1) so that the error is reduced. out The specific method of the update process for updating is not particularly limited, but for example, a gradient descent method or the like may be used.

[0034] The above-mentioned learning process is executed using sensor data acquired during a predetermined preparation period. The preparation period is a period during which no person (foot F, etc.) is present within the detection range of the kick sensor 11, and is preferably a period prior to and as close as possible to the timing at which a kicking motion is performed.

[0035] Fig. 5 is a diagram showing an example of a preparation period T in the embodiment. Fig. 5 illustrates a situation in which a specific person P gradually approaches the vehicle 1. Here, the specific person P is assumed to be a person carrying a smart key 51 for driving the vehicle 1, and the vehicle 1 is assumed to have a function of estimating the distance between the vehicle 1 and the person P (smart key 51) based on an electromagnetic signal transmitted from the smart key 51.

[0036] The preparation period T may be, for example, a period from when the person P enters an area within a distance D1 (an example of a first distance) from the vehicle 1 until when the person P reaches a position at a distance D2 (an example of a second distance) from the vehicle 1 that is smaller than the distance D1. By setting the preparation period T in this manner, it is possible to generate a model M that corresponds to the environment (e.g., weather, etc.) immediately before the person P performs a kicking motion. Furthermore, according to this embodiment, the model M is generated by reservoir computing, so that the model M can be generated quickly and with a low computational load.

[0037] 6 is a flowchart showing an example of processing for generating a model M in the detection system S1 of the embodiment. The generation unit 102 determines whether the vehicle 1 is stopped (S101), and ends this routine if the vehicle 1 is not stopped (S101: No). If the vehicle 1 is stopped (S101: Yes), the generation unit 102 determines whether the person P carrying the smart key 51 has entered an area within a distance D1 from the vehicle 1 (S102).

[0038] If the person P has not entered within the distance D1 (S102: No), this routine ends. If the person P has entered within the distance D1 (S102: Yes), the generation unit 102 determines that the preparation period T has started, and executes the learning process as described above using the sensor data output from the kick sensor 11 during the preparation period T to generate the model M (S103).

[0039] Thereafter, the generation unit 102 determines whether the person P has reached a position at a distance D2 from the vehicle 1 (S104), and if the person P has not reached the distance D2 (S104: No), executes the processes from step S102 onwards again. At this time, if the person P remains within the distance D1 (S102: Yes), the learning process continues (S103), and if the person P has left the distance D1 (S102: No), this routine ends.

[0040] If the person P has reached the distance D2 (S104: Yes), the generation unit 102 determines that the preparation period T has ended, and ends the generation of the model M.

[0041] The determination unit 103 of this embodiment determines whether or not the person P has performed a kicking motion based on the predicted value output from the model M generated by the generation unit 102 during the preparation period T as described above (S105).

[0042] 7 is a diagram showing an example of a method for determining whether or not a kicking motion has occurred in the embodiment, and illustrates the process performed by the determination unit 103 in the estimation stage for determining whether or not a kicking motion has actually occurred.

[0043] In the estimation stage, the determination unit 103 determines whether or not a kicking motion has occurred based on a difference Δu between a predicted value y(t) obtained by inputting an input value u(t) of sensor data corresponding to a first time t to the model M and an input value u(t+1) of sensor data corresponding to a second time t+1. For example, if the difference Δu is greater than a threshold, it is determined that a kicking motion has been performed, and if the difference Δu is equal to or less than the threshold, it is determined that a kicking motion has not been performed. Since the predicted value y(t) output from the model M is a value corresponding to a situation in which a kicking motion has not been performed, it can be estimated that a kicking motion has been performed when the difference Δu is large.

[0044] 8 is a flowchart showing an example of a process for determining the presence or absence of a kicking motion in the detection system S1 of the embodiment. The determination unit 103 inputs an input value u(t) of the sensor data corresponding to a first time t to the input layer 201 of the model M to obtain a predicted value y(t) obtained from the model M and an input value (correct value) u(t+1) of the sensor data corresponding to a second time t+1, calculates a difference Δu therebetween (S201), and determines whether the difference Δu is greater than a threshold value (S202). If the difference Δu is greater than the threshold value (S202: Yes), the determination unit 103 determines that a kicking motion has been performed (S203), and if the difference Δu is not greater than the threshold value (S202: No), determines that a kicking motion has not been performed (S204).

[0045] As described above, according to this embodiment, the model M is generated by a learning process that is performed at high speed using sensor data acquired during the preparation period T immediately before entering the estimation stage. Therefore, the predicted value y(t) output from the model M is a value that takes into account the environment (e.g., weather, etc.) during the preparation period T. This reduces the influence of the environment in the estimation stage, making it possible to determine with high accuracy whether or not a kicking motion has been performed.

[0046] A program for causing a computer (such as the determination ECU 12) to execute a process for realizing the function of the detection system S1 of the above embodiment may be provided as a computer program product by being stored in a computer-readable storage medium such as a CD-ROM, a CD-R, a memory card, a DVD (Digital Versatile Disk), or a flexible disk (FD) in an installable or executable format file. The program may also be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network. The program may also be provided or distributed via a network such as the Internet.

[0047] Although the embodiment of the present invention has been described above, the above embodiment is presented as an example and is not intended to limit the scope of the invention. This new embodiment can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the gist of the invention. This embodiment and its modifications are included in the scope and gist of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0048] 1...vehicle, 2...vehicle body, 3...tail door, 4...rear bumper, 5...sliding door, 11...kick sensor, 12...determination ECU, 21...door control ECU, 22...lock mechanism, 23...driving mechanism, 51...smart key, 101...acquisition unit, 102...generation unit, 103...determination unit, 111...learning processing unit, 201...input layer, 202...reservoir layer, 203...output layer, F...foot, M...model, P...person, S...door opening / closing system, S1...detection system, T...preparation period

Claims

1. an acquisition unit that acquires sensor data output from a sensor; a generation unit that executes a learning process for a neural network using the sensor data acquired during a predetermined preparation period to generate, for each preparation period, a model in which the sensor data corresponding to a first time is input and a predicted value of the sensor data corresponding to a second time that is a predetermined time after the first time is output; a determination unit that determines whether or not a predetermined event has occurred based on a difference between the predicted value output from the model and the actual sensor data corresponding to the second time; A detection system comprising:

2. The neural network includes a reservoir layer and an output layer that outputs the predicted value by weighting an output from the reservoir layer with a predetermined output connection weight, The learning process includes a reservoir computing for updating the output connection weights based on an error between the predicted value and a correct answer value. The detection system of claim 1 .

3. the sensor data changes in response to a movement of a person occurring within a detection range of the sensor; the preparation period is a period during which the person is not present within the detection range, The event is a predetermined action performed by the person within the detection range. The detection system of claim 1 .

4. the sensor is a capacitance sensor installed at a position corresponding to an opening / closing mechanism of the movable body, The motion is a kicking motion in which the person puts his / her foot in and out of a space below the moving object. The detection system of claim 3 .

5. the preparation period includes a period from when a person carrying a key for operating the moving body enters an area within a first distance from the moving body until when the person reaches a position away from the moving body by a second distance that is smaller than the first distance. The detection system of claim 4.

Images