Edge Devices and Distributed Systems

Abstract

To provide an edge device capable of contributing to improvement of safety by efficiently collecting situation data at a time when an unexpected event occurs in an automatic working body, and a distribution system.SOLUTION: An edge device comprises: a sensor provided in an automatic working body; a recognition section which recognizes an object based on input data from the sensor; a determination section which makes a determination with respect to a recognition result in the recognition section; a drive mechanism control section for controlling a drive mechanism based on a determination result in the determination section; an unexpected event decision section for deciding whether or not an unexpected event occurs based on information from the recognition section and the determination section; a data shaping section by which, in a case where it is decided that an unexpected event occurs, the recognition result in the recognition section or the input data used for the recognition in the recognition section and the determination result in the determination section or a calculation history to the determination result are shaped as communication data; and a communication section for transmitting the communication data shaped by the data shaping section to a cloud server.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an edge device and a distributed system. [Background technology]

[0002] There are known electronic systems that assist in controlling the drive mechanisms of autonomous vehicles, robots, and other autonomous operating bodies. For example, Patent Document 1 describes a vehicle control device that estimates disturbances, such as crosswinds, that affect the vehicle, and changes the content of driving assistance based on the results of the estimated disturbances. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-168955 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology described in Patent Document 1 is only concerned with determining whether a disturbance has an effect on the vehicle's operation and providing driving assistance accordingly, and is not intended to collect situational data during a disturbance.

[0005] An object of the present invention is to provide an edge device and a distributed system that can efficiently collect situation data when an unexpected event occurs in an automated operating device, thereby contributing to improved safety. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems, the edge device of the present invention is connected via a network to a cloud server that collects data regarding an automated operating body and assists in controlling a drive mechanism of the automated operating body, and comprises a sensor provided on the automated operating body, a recognition unit that recognizes objects based on input data from the sensor, a judgment unit that makes a judgment on the recognition result by the recognition unit, a drive mechanism control unit that controls the drive mechanism based on the judgment result by the judgment unit, an unexpected event judgment unit that judges whether or not an unexpected event has occurred based on information from the recognition unit and the judgment unit, a data formatting unit that, when it is judged that an unexpected event has occurred, formats the recognition result by the recognition unit or the input data used for recognition by the recognition unit, as well as the judgment result by the judgment unit or the calculation history leading up to the judgment result, as communication data, and a communication unit that transmits the communication data formatted by the data formatting unit to the cloud server. Effect of the Invention

[0007] According to the present invention, it is possible to provide an edge device and a distributed system that can efficiently collect situation data when an unexpected event occurs in an automated operating device, thereby contributing to improving safety.

[0008] Problems, configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing an overall configuration of a distributed system according to a first embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram showing the hardware configuration of a computer constituting a diagnostic cloud server. [Diagram 3] FIG. 2 is a diagram showing the flow of data in the distributed system shown in FIG. 1. [Figure 4] A conceptual diagram showing an example of an autonomous vehicle service that uses a distributed system. [Diagram 5] FIG. 1 shows the hardware configuration of a driving assistance device in an autonomous vehicle as an example of an edge device. [Figure 6] 4 is a flowchart showing a process in the ECU. [Figure 7] A table showing the types of unexpected events that can occur in autonomous vehicles in relation to other vehicles. [Figure 8A] FIG. 13 is a schematic diagram showing the movement of a vehicle when another vehicle cuts in as an example of an unexpected event. [Figure 8B] 8B is a graph showing criteria for classifying events when another vehicle cuts in as in FIG. 8A; [Figure 9] 11 is a flowchart showing an example of processing in a diagnostic cloud server. [Figure 10] FIG. 11 is a diagram showing the overall configuration of a distributed system according to a second embodiment of the present invention. [Figure 11] FIG. 11 is a diagram showing the overall configuration of a distributed system according to a third embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The embodiment is an example for explaining the present invention, and appropriate omissions and simplifications have been made to clarify the explanation. The present invention can be implemented in various other forms. Unless otherwise specified, each component may be singular or plural.

[0011] In the embodiments, the processing performed by executing a program may be described. Here, the computer executes the program using a processor (e.g., CPU, GPU), and performs the processing defined by the program while using storage resources (e.g., memory) and interface devices (e.g., communication ports). Therefore, the subject of the processing performed by executing the program may be the processor. Similarly, the subject of the processing performed by executing the program may be a controller, device, system, computer, or node having a processor. The subject of the processing performed by executing the program may be a calculation unit, and may include a dedicated circuit that performs specific processing. Here, the dedicated circuit is, for example, an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), or a CPLD (Complex Programmable Logic Device).

[0012] The program may be installed on the computer from a program source. The program source may be, for example, a program distribution server or a computer-readable storage medium. When the program source is a program distribution server, the program distribution server may include a processor and a storage resource for storing the program to be distributed, and the processor of the program distribution server may distribute the program to be distributed to other computers. In the embodiments, two or more programs may be realized as one program, and one program may be realized as two or more programs. EXAMPLES

[0013] (Overall configuration of distributed system) 1 is a diagram showing the overall configuration of a distributed system according to a first embodiment of the present invention. The distributed system according to this embodiment includes an edge device 1, a diagnostic cloud server 2, and a management server 3. The edge device 1 is connected to the diagnostic cloud server 2 and the management server 3 via a network 20. The network 20 is a two-way communication network represented by, for example, a mobile phone communication network, the Internet (including Ethernet), and the like.

[0014] (Edge device configuration) The edge device 1 is a device that assists in the control of a drive mechanism of an automatic operating body, and includes a sensor 7, a recognition unit 4, a judgment unit 5, a drive mechanism control unit 6, a first memory 9, a second memory 10, an unexpected event determination unit 11, a data shaping unit 12, and a communication unit 8. Examples of automatic operating bodies include moving bodies such as vehicles, drones, and robots, and equipment such as robot arms, machine tools, and numerically controlled lathes. When the automatic operating body is a moving body, the drive mechanism is an engine, a motor, or the like, and when the automatic operating body is equipment, the drive mechanism is an actuator such as a motor or hydraulics. Here, automatic operation includes automatic driving.

[0015] The sensor 7 is a camera, radar, etc., installed in the automated operating body. The recognition unit 4 recognizes what type of object the sensed target is based on the input data from the sensor 7, and converts it into object data. The judgment unit 5 makes a judgment on the object data that is the recognition result in the recognition unit 4, and determines the next operation of the automated operating body, i.e., the control content of the drive mechanism. The drive mechanism control unit 6 controls the operation of the drive mechanism based on the judgment result in the judgment unit 5.

[0016] The first memory 9 holds object data that is the recognition result in the recognition unit 4, and / or input data (e.g., image data, distance data, etc., RAW data as output from the sensor 7) used for recognition in the recognition unit 4. The second memory 10 holds the judgment result in the judgment unit 5, and / or the calculation history leading up to the judgment result.

[0017] The unexpected event determination unit 11 determines whether or not an unexpected event has occurred based on information from the recognition unit 4 and the judgment unit 5. Specifically, the unexpected event determination unit 11 determines whether or not an unexpected event has occurred by comparing calculation history data leading up to the judgment result in the judgment unit 5 with a predetermined threshold value for the object data recognized by the recognition unit 4. When it is determined that an unexpected event has occurred, the unexpected event determination unit 11 issues a trigger signal to the first memory 9 and the second memory 10 to instruct them to hold the data.

[0018] Here, examples of unexpected events will be described. The first example is when an object can be recognized based on input data from the sensor 7, but the shape is difficult to identify based on the previously learned results (recognition may be successful). The second example is when multiple objects are recognized based on input data from the sensor 7, but the combination is unexpected (for example, when an autonomous vehicle recognizes a stop sign on a highway). The third example is when the shape of an object ahead (for example, a vehicle ahead of the autonomous vehicle or an oncoming vehicle) appears to be missing due to the lighting conditions. The fourth example is when an unexpected event occurs due to the relationship with the surrounding environment (for example, when a vehicle running alongside the autonomous vehicle makes a sudden movement). Note that unexpected events are not limited to these examples.

[0019] Furthermore, a rule base that stores normal operation scenario data and sensing data (e.g., shape data) of recognizable objects may be provided in the unexpected event determination unit 11. The unexpected event determination unit 11 can determine an unexpected event by sequentially checking input data from the recognition unit 4 and the judgment unit 5 against this rule base.

[0020] When the data formatting unit 12 acquires each piece of data held in the first memory 9 and the second memory 10, it formats the data as communication data by converting the data into a predefined format and outputs it to the communication unit 8.

[0021] The communication unit 8 transmits the communication data reformed by the data reforming unit 12 to the diagnostic cloud server 2 via the network 20.

[0022] (Configuring the diagnostic cloud server) The diagnostic cloud server 2 is composed of one or more computers existing on the network 20. Note that instead of a cloud server, a server composed of one or more locally existing computers may be adopted. The diagnostic cloud server 2 includes a data classification unit 21 and a learning data generation unit 22.

[0023] The data classification unit 21 classifies the type of unexpected events from each of the communication data received from the edge device 1, i.e., the input data (raw data), the object data, the judgment result in the judgment unit 5, and the calculation history leading up to the judgment result. Furthermore, the data classification unit 21 judges whether learning is necessary based on the classification result, and outputs data that is judged to require new learning to the learning data generation unit 22.

[0024] The unexpected event determination unit 11 or the data reforming unit 12 of the edge device 1 may perform a primary classification and labeling of the type of unexpected event. In that case, the data classification unit 21 of the diagnostic cloud server 2 may only determine whether learning is necessary according to the label, or may perform a more detailed classification based on the re-input data, and in addition to determining whether learning is necessary, may extract and label data parts that are more effective as learning data. Note that a human may also be involved in the labeling.

[0025] The learning data generation unit 22 generates a learning data format and learning label data based on the output result from the data classification unit 21, the input data (raw data), the object data, and the like.

[0026] FIG. 2 is a diagram showing a hardware configuration of a computer 400 that constitutes the diagnostic cloud server 2. As shown in FIG.

[0027] The computer 400 comprises a processor 401, a memory 402, an external storage device 403, an audio output device 404, a biometric information input device 405, an input device 406, an output device 407, and a communication device 408, which are connected via a data bus 409.

[0028] The processor 401 is composed of a CPU, a GPU, an FPGA, etc., and controls the entire computer 400. The memory 402 is a main storage device such as a RAM (Random Access Memory). The external storage device 403 is a non-volatile storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a flash memory that can store digital information.

[0029] The audio output device 404 includes a speaker, etc. The biometric information input device 405 includes a camera, a gaze input device, a microphone, etc. The input device 406 includes a keyboard, a mouse, a touch panel, etc. The output device 407 includes a display, a printer, etc.

[0030] The communication device 408 is composed of a NIC (Network Interface Card) or the like. The communication device 408 communicates with other devices connected to the same network by at least one of wired communication and wireless communication. For this communication, packet communication by TCP / IP (Transmission Control Protocol / Internet Protocol) is adopted, but this is not limited thereto, and communication by other protocols such as UDP (User Datagram Protocol) may also be adopted.

[0031] The hardware configuration of the computer 400 is not limited to the above example, and some of the above components may be omitted or other components may be included. The computer 400 may be any of various information processing devices, such as a server computer, a personal computer, a notebook computer, a tablet computer, a smartphone, or a television device.

[0032] The computer 400 can store and externally load programs such as an OS (Operating System), middleware, and application programs, and can execute various types of processing by the processor 401 executing these programs.

[0033] (Administration Server Configuration) The management server 3 is assumed to be a computer owned by a company (hereinafter, sometimes simply referred to as a manufacturing company) that designs and manufactures automated operating bodies and edge devices 1. The manufacturing company manages edge data related to the edge devices 1 for the development and maintenance of its own products. The management server 3 generates data for updating the functions of the edge devices 1 by executing a learning process based on the learning data, and distributes it to the edge devices 1. The learning data may also be used by the manufacturing company for the development of new products.

[0034] The learning unit 31 inputs learning data generated by the diagnostic cloud server 2 and performs new learning based on the processing contents of the edge device 1 managed by the management server 3 (such as the programs and setting parameters of the recognition unit 4, judgment unit 5, and unexpected event judgment unit 11).

[0035] The function update data generation unit 32 converts the learning result information (program, setting parameters, neural network learning coefficient data, etc.) executed by the learning unit 31 into function update data in a format that can be written to the edge device 1, and distributes it to the edge device 1 via the network 20.

[0036] The distributed function update data is received by the communication unit 8 of the edge device 1 and written to at least one of the recognition unit 4, the judgment unit 5, and the unexpected event determination unit 11. Furthermore, learning using the learning data and creation of the function update data may be performed manually, without relying on the management server 3, or may be performed by the diagnostic cloud server 2.

[0037] (Data flow in distributed systems) Fig. 3 is a diagram showing the flow of data in the distributed system shown in Fig. 1. In Fig. 3, from the left, the operating states or processing contents of the edge device 1, the diagnostic cloud server 2, and the management server 3 are shown. During the product operation of the edge device 1, data is input from the sensor 7, an object is recognized by the recognition unit 4, and a judgment is made by the judgment unit 5. The judgment result by the judgment unit 5 is sent to the drive mechanism control unit 6 and used to control the drive mechanism (actuator). These flows are the main functions of the edge device 1, and these flows are repeated in normal operating conditions.

[0038] While this flow is repeated, the unexpected event determination unit 11 performs the process of determining an unexpected event, and when it is determined that an unexpected event has occurred, the unexpected event determination unit 11 issues a trigger signal for instructing the first memory 9 and the second memory 10 to hold the respective data temporarily. After that, the data reforming unit 12 converts the data held in each memory into a data format for communication, and the communication unit 8 transmits the data to the diagnostic cloud server 2 via the network 20.

[0039] The diagnostic cloud server 2 classifies the received data and generates learning data, and transmits the learning data to the management server 3. The management server 3 learns the situation when an unexpected event occurs based on the received learning data, generates function update data for the recognition unit 4, judgment unit 5, etc. of the edge device 1, and transmits it to the edge device 1. In addition, the situation data when an unexpected event occurs and the learning data may be fed back to the development of next-generation products in the manufacturer of the edge device 1.

[0040] (Example of an autonomous vehicle service using a distributed system) FIG. 4 is a conceptual diagram showing an example of an autonomous vehicle service using a distributed system. A product purchaser 41 (here, a purchaser of an autonomous vehicle) transmits vehicle status data to a data management company 42 periodically or at any time. The data management company 42 owns a diagnostic cloud server 2, and analyzes and converts the data received by the diagnostic cloud server 2. The data management company 42 transmits image data, which is information on the surrounding environment, data on the recognition results and judgment results (distance to an oncoming vehicle and relative speed, etc.), the presence or absence of an abnormality in the vehicle's operation, and data on the state inside the vehicle when an abnormality occurs, to a system design and manufacturing company 43. These data are used by the system design and manufacturing company 43 for designing to improve safety and reliability in the next-generation system. The system design and manufacturing company 43 generates function update data for a vehicle in operation based on the collected data, and transmits the generated function update data to a data distribution company 44. However, the data distribution company 44 may be included in either or both of the system design and manufacturing company 43 and the data management company 42. The data distribution company 44 transmits the function update data to the product purchaser 41 via a network. In addition, for example, function update data may be sent to a product operation operator / service provider 45 such as a passenger transport operator or a MaaS (Mobility as a Service) provider, and then sent to vehicles managed and operated by the product operation operator / service provider 45.

[0041] (Example of edge devices installed in self-driving cars) Fig. 5 is a diagram showing a hardware configuration of a driving assistance device in an autonomous vehicle as an example of an edge device. When the autonomous operating body is an autonomous vehicle (mobile body), the edge device is mounted on the vehicle 13 as a driving assistance device 1a. As shown in Fig. 4, the driving assistance device 1a (edge ​​device) includes an ECU 14 (Electronic Control Unit), a communication unit 8, sensors (camera 7a, radar 7b, etc.), and a drive mechanism control unit (engine control unit 6a that controls the engine 15a, steering control unit 6b that controls the steering 15b, etc.).

[0042] The ECU 14 has a recognition unit 4, a judgment unit 5, a first memory 9, a second memory 10, an unexpected event judgment unit 11, and a data shaping unit 12. The recognition unit 4 recognizes objects such as other vehicles and road signs based on image data acquired by the camera 7a. The judgment unit 5 judges how the vehicle should move, taking into account the position and speed of the object recognized by the recognition unit 4.

[0043] 6 is a flowchart showing the processing in the ECU. First, the ECU 14 acquires input data such as image data from a sensor such as the camera 7a (step S601). Next, the recognition unit 4 recognizes an object based on the acquired input data (step S602). The judgment unit 5 calculates the distance from the recognized object (e.g., the distance between the vehicle and another vehicle) and the relative speed (e.g., the cut-in speed) using distance information from a sensor such as the radar 7b, and judges the control content for the drive mechanism control unit 6 (step S603). After that, the unexpected event judgment unit 11 compares the calculation history (distance and relative speed on which the judgment result was based, etc.) up to the judgment result of the judgment unit 5 with various thresholds of the safety evaluation boundary conditions (vehicle conditions, pedestrian conditions, weather conditions, road surface conditions, etc.) stored in the risk database 60 (step S604). If the safety evaluation boundary is exceeded, the unexpected event determination unit 11 determines that an unexpected event has occurred, and issues a trigger signal to instruct the first memory 9 and the second memory 10 to hold data (step S605). The data reforming unit 12 extracts only a portion of the data from the large amount of RAW data, such as image data, held in the first memory 9, for example, taking into consideration the communication load, and reforms the data as communication data (step S606). The reformed communication data is transmitted to the diagnostic cloud server 2 by the communication unit 8.

[0044] (Examples of unexpected events in self-driving cars) Figure 7 is a table showing the types of unexpected events that occur in autonomous vehicles due to their relationship with other vehicles. Sudden unexpected events (sometimes called traffic disturbances) due to the surrounding environment in autonomous vehicles are caused by the road shape, the behavior of the vehicle itself and other vehicles, and the surrounding conditions. Examples of road shapes include cases where the shape of the lanes is difficult to understand (when they are difficult to see or easy to cause illusions), when there is a merging road, when there is a branching road, when a three-dimensional road is recognized as having an unexpected shape, etc. Examples of the behavior of the vehicle itself and other vehicles include cases where a vehicle traveling alongside the vehicle that has been maintaining its lane or a vehicle ahead suddenly changes lanes. Examples of surrounding conditions include a sudden cut-in by a nearby vehicle, a cut-out by a vehicle ahead, sudden acceleration or deceleration, etc.

[0045] FIG. 8A is a schematic diagram showing the movement of another vehicle cutting in as an example of an unexpected event, and FIG. 8B is a graph showing criteria for classifying events when another vehicle cuts in as in FIG. 8A.

[0046] 8A shows an event in which another vehicle 52 cuts in from the right lane in the direction of travel with the own vehicle 51 as the reference, into the left lane in which the own vehicle 51 is traveling. The own vehicle 51, which is an autonomous vehicle, constantly observes the other vehicle 52 using a sensor 7 installed in front, and travels by detecting the relative speed and distance between the own vehicle 51 and the other vehicle 52. The judgment unit 5 in the ECU 14 of the own vehicle 51 calculates the positional relationship with the other vehicle 52 from the speed of the own vehicle 51, in addition to the data on the relative speed and distance between the vehicles, and judges whether there is a risk of collision.

[0047] In FIG. 8B, the horizontal axis represents the distance L between the vehicle 51 and the other vehicle 52. mo ,The vertical axis is the speed V of the other vehicle 52 when it cuts in. c, respectively. Collision time = vehicle distance / relative speed, and curves serving as thresholds for determining the possibility of a collision are shown in stages in FIG. 8B. The dotted line in FIG. 8B indicates the curve of the safety assessment boundary, and if the vehicle 51 passes beyond this curve and enters the inner area (lower left), the vehicle will execute deceleration and stopping control to avoid a collision. The solid lines in FIG. 8B indicate the curves that lead to rear and side collisions, respectively, and if the vehicle 51 passes beyond these curves and enters the inner area (lower left), this means that the vehicle 51 will collide with the other vehicle 52.

[0048] In this embodiment, an unexpected event that may result in a collision is used as a trigger, and regardless of whether there is an abnormality in the vehicle's operation, input data from the sensor 7 and the like are transmitted to the diagnostic cloud server 2, which then collects vehicle information. The diagnostic cloud server 2 classifies the received data. Here, in the case of an event in which another vehicle cuts in, the event can be classified into the following four events.

[0049] First, (event 1) is in the area outside (upper right) the safety evaluation boundary in Fig. 8B, and can be handled without any problem even if another vehicle cuts in. Since (event 1) is an expected event, it is not essential to transmit the input data, etc. for this event to the diagnostic cloud server 2, but the data may be transmitted to improve the object recognition accuracy and treated as learning data. In particular, by transmitting data at the time when the possibility of a collision (the curve of the safety evaluation boundary shown by the dotted line in Fig. 8B) is approached, even if avoidance action is not taken, it is possible to collect information on the margin of the threshold, etc., which can lead to the suppression of sudden avoidance action or sudden stopping.

[0050] Next, (event 2) is in the area between the safety assessment boundary in Figure 8B and the curve (rear) that leads to a collision, and can be avoided with plenty of time even if another vehicle cuts in. Since (event 2) is an unexpected event, the input data for that event is transmitted to the diagnostic cloud server 2, where it is treated as learning data and used to reset each threshold value. Note that the threshold values ​​are reset either automatically by the diagnostic cloud server based on the difference from the expected value, or manually.

[0051] Furthermore, (event 3) is in the area between the safety assessment boundary and the curve (rear) that leads to a collision in Fig. 8B, particularly in the area close to the latter curve, and although it can be avoided by autonomous driving, it is an event that requires a human to apply the brakes suddenly, etc. Since (event 3) is also an unexpected event, the input data, etc. for this event are transmitted to the diagnostic cloud server 2 and treated as learning data. The diagnostic cloud server 2 also analyzes the recognition results in the vehicle, and if necessary, reviews the labeling and generates additional learning data.

[0052] And (event 4) is in the area inside (lower left) of the curve (rear) that leads to the collision in Figure 8B, and since it may be difficult to avoid, the autonomous vehicle activates the fail operation and safely stops. Since this (event 4) is also an unexpected event, the input data etc. for this event are sent to the diagnostic cloud server 2 and treated as learning data. The diagnostic cloud server 2 also analyzes the recognition results in the vehicle, reviews the labeling, and generates additional learning data. The additional learning data is sent to the management server 3 and used for function updates.

[0053] In this embodiment, learning about unexpected events enables avoidance of danger from nearby events and avoidance action with sufficient time before a dangerous situation occurs, thereby further improving the reliability and safety of self-driving cars. Note that this embodiment is premised on the fact that the classification of events as described above is performed by the data classification unit 21 of the diagnostic cloud server 2, but it may also be performed by the unexpected event determination unit 11 or the judgment unit 5 of the edge device.

[0054] 9 is a flowchart showing an example of processing in the diagnostic cloud server. First, when the diagnostic cloud server 2 receives data determined to be an unexpected event from the edge device 1 via the network 20 (step S901), the data classification unit 21 classifies the event (step S902). Furthermore, when an unexpected event of a predetermined classification occurs, the learning data generation unit 22 generates additional learning data based on data such as images collected from the edge device 1 (step S903). For example, in the case of an event in which another vehicle cuts in as described above, additional learning data is generated when an unexpected event classified as (event 3) or (event 4) occurs.

[0055] Thereafter, the diagnostic cloud server 2 extracts input data from the sensors 7, for example, data on the movement (speed, direction, etc.) of the object obtained from the front, side, and rear sensors 7 (step S904). Furthermore, the diagnostic cloud server 2 creates function update data using the extracted data, and the function update data is distributed to the edge device 1 and reflected in parameters of the recognition unit 4, etc. (step S905). When creating the function update data, for example, it is evaluated whether deceleration was effective for cutting in, whether the deceleration rate was appropriate, whether the lane change was effective, etc. Furthermore, when the function of the edge device 1 is updated, for example, the recognition rate of the side of another vehicle cutting in is improved.

[0056] In the example of FIG. 9, the diagnostic cloud server 2 executes the learning and generates the function update data, but as described above, these may be performed by the management server 3.

[0057] According to the distributed system described above, the edge device 1 and computers such as the diagnostic cloud server 2 cooperate with each other to efficiently collect situation data when various unexpected events occur in an automated operating body. In particular, when it is determined that an unexpected avoidance action or change in the surrounding environment has occurred, the edge device 1 transmits sensor data and the like at the time of the determination to the diagnostic cloud server 2, which reduces the amount of collected data. Then, it becomes possible to continuously improve safety and reliability by analyzing the situation data of the unexpected event that has occurred and performing additional learning and adjusting parameters that reflect the analysis results. EXAMPLES

[0058] 10 is a diagram showing the overall configuration of a distributed system according to a second embodiment of the present invention. Unlike the edge device 1 of the first embodiment, the edge device 1 of the second embodiment further includes a driving scene identification unit 101 that identifies a driving scene of an automated operating body, and a position detection unit 102 that acquires position information of the automated operating body.

[0059] When the autonomous operating body is an autonomous vehicle, the position detection unit 102 acquires current location data using data from a global positioning system (GPS) or other artificial satellites. When the driving scene identification unit 101 receives the current location data acquired by the position detection unit 102, it identifies, for example, the type of road (general road, expressway, highly congested road, etc.) as the scene on which the autonomous vehicle is traveling, and sends the result to the unexpected event determination unit 11. The driving scene identification unit 101 has a database of map information, and identifies the driving scene based on the map data and the current location data. Note that the edge device 1 does not need to have the database of map information. For example, the edge device 1 may receive the map information via the network 20, or may receive data for identifying the driving scene together with the current location data by the position detection unit 102.

[0060] The unexpected event determination unit 11 of this embodiment determines whether an unexpected event has occurred under different conditions for each driving scene identified by the driving scene identification unit 101. For example, when an autonomous vehicle is driving on a highway, it is more important that the relationship with another vehicle is unexpected than that the recognition of road signs is unexpected. For this reason, when an autonomous vehicle is driving on a highway, the latter case is more likely to be determined as an unexpected event than the former case, and data of the latter case can be preferentially transmitted to the diagnostic cloud server 2. In addition, it is possible to set conditions such as increasing the importance of relationships with people when an autonomous vehicle is driving in a city, and increasing the importance of recognition of road signs when an autonomous vehicle is driving in a suburban area.

[0061] According to this embodiment, the communication load is reduced and data collection efficiency is improved compared to the case where all data is transmitted to the diagnostic cloud server 2. If the communication environment has room for capacity, the diagnostic cloud server 2 may specify the driving scene using the current location data, etc., and classify the events to be collected according to the importance of each driving scene.

[0062] In addition, since the driving scene identification unit 101 in this embodiment identifies the driving scene using the current location data acquired by the position detection unit 102, the driving scene can be identified with high accuracy, and as a result, the accuracy of data collection is improved. However, even if the position detection unit 102 is not present, the driving scene may be identified based on the driving speed acquired by the sensor 7. For example, when the driving speed is 100 km / h or more, the driving scene identification unit 101 can identify that the vehicle is traveling on an expressway. EXAMPLES

[0063] 11 is a diagram showing an overall configuration of a distributed system according to a third embodiment of the present invention. Unlike the edge device 1 of the first embodiment, the edge device 1 of the present embodiment further includes a third memory 103 for temporarily storing communication data shaped by the data shaping unit 12.

[0064] The edge device 1 of this embodiment has a function of switching between a mode in which communication data from the data reforming unit 12 is immediately transmitted to the diagnostic cloud server 2 without passing through the third memory 103, and a mode in which communication data from the data reforming unit 12 is temporarily stored via the third memory 103 and then transmitted to the diagnostic cloud server 2. As a result, data can be reliably transmitted to the diagnostic cloud server 2 even when, for example, an autonomous vehicle, which is an autonomous operating body, travels in a place with poor radio wave conditions such as inside a tunnel or in a mountainous area.

[0065] The present invention is not limited to the above-described embodiments, and various modifications are possible. For example, the above-described embodiments are described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those having all of the configurations described. In addition, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, or to add to it. [Explanation of symbols]

[0066] 1...edge device, 1a...driving assistance device, 2...diagnostic cloud server, 3...management server, 4...recognition unit, 5...judgment unit, 6...drive mechanism control unit, 6a...engine control unit, 6b...steering control unit, 7...sensor, 7a...camera, 7b...radar, 8...communication unit, 9...first memory, 10...second memory, 11...unexpected event judgment unit, 12...data reforming unit, 13...vehicle, 14...ECU, 15a...engine, 15b...steering, 21...data classification unit, 22...learning data generation unit, 31...learning unit, 32...function update data generation unit, 51...own vehicle, 52...other vehicle, 60...risk database, 101...driving scene identification unit, 102...position detection unit, 103...third memory, 400...computer, 401...processor, 402...memory, 403...external storage device, 404...audio output device, 405...biometric information input device, 406...input device, 407...output device, 408...communication device, 409...data bus

Claims

1. An edge device that is connected to a cloud server that collects data on an automated operating body via a network and assists in control of a drive mechanism of the automated operating body, A sensor provided on the automatic operating body; a recognition unit that recognizes an object based on input data from the sensor; a judgment unit for making a judgment on a recognition result by the recognition unit; a drive mechanism control unit that controls the drive mechanism based on a result of the determination by the determination unit; an unexpected event determination unit that determines whether or not an unexpected event has occurred based on information from the recognition unit and the determination unit; a data formatting unit that formats, when it is determined that an unexpected event has occurred, a recognition result by the recognition unit, or the input data used for recognition by the recognition unit, and a judgment result by the judgment unit, or a calculation history leading to the judgment result, as communication data; a communication unit that transmits the communication data shaped by the data shaping unit to the cloud server; Equipped with The unexpected event is a disturbance that affects the operation of the automatic operating body, and includes an event that can be avoided without the automatic operating body taking sudden danger avoidance action.

2. The edge device according to claim 1, When it is determined that an unexpected event has occurred, the data formatting unit formats the recognition result by the recognition unit, the input data used for recognition by the recognition unit, the judgment result by the judgment unit, and the calculation history leading to the judgment result into communication data.

3. The edge device according to claim 2, a first memory for storing a recognition result by the recognition unit and the input data used for the recognition by the recognition unit; A second memory for storing a judgment result by the judgment unit and a calculation history leading to the judgment result, When it is determined that an unexpected event has occurred, the unexpected event determination unit issues an instruction to the first memory and the second memory to retain data.

4. The edge device according to claim 3, When it is determined that an unexpected event has occurred, the data reforming unit extracts a portion of the input data stored in the first memory and reforms it as communication data.

5. The edge device according to claim 1, Further comprising a driving scene identification unit that identifies a driving scene of the automated operating body, The unexpected event determination unit is an edge device that determines whether an unexpected event has occurred under different conditions for each driving scene identified by the driving scene identification unit.

6. The edge device according to claim 5, The edge device further includes a location detection unit that acquires current location data of the automatic operating body, The driving scene identification unit is an edge device that identifies a driving scene based on the current location data and map data.

7. The edge device according to claim 1, Further comprising a third memory for temporarily storing the communication data shaped by the data shaping unit, An edge device that is switchable between a mode in which the communication data is transmitted to the cloud server via the third memory and a mode in which the communication data is transmitted to the cloud server without passing through the third memory.

8. The edge device according to claim 1, The unexpected event includes not only areas that may lead to a collision, but also areas that do not lead to a collision and that exceed a safety assessment boundary.

9. An edge device that assists in controlling a drive mechanism of an automatic operating body; A diagnostic cloud server that generates learning data based on the data received from the edge device; a management server that generates function update data for the edge device using a result of learning based on learning data generated by the diagnostic cloud server, The edge device includes: A sensor provided on the automatic operating body; a recognition unit that recognizes an object based on input data from the sensor; a judgment unit for making a judgment on a recognition result by the recognition unit; a drive mechanism control unit that controls the drive mechanism based on a result of the determination by the determination unit; an unexpected event determination unit that determines whether or not an unexpected event has occurred based on information from the recognition unit and the determination unit; a data formatting unit that formats, when it is determined that an unexpected event has occurred, a recognition result by the recognition unit, or the input data used for recognition by the recognition unit, and a judgment result by the judgment unit, or a calculation history leading to the judgment result, as communication data; A communication unit that transmits the communication data shaped by the data shaping unit to the diagnostic cloud server; Equipped with A distributed system in which the unexpected event is a disturbance that affects the operation of the automated operating body and includes an event that can be avoided without the automated operating body taking sudden danger avoidance action.

10. 10. The distributed system according to claim 9, The diagnostic cloud server includes: a data classification unit that classifies a type of unexpected event from communication data received from the edge device; a learning data generating unit that generates learning data based on an output result from the data classifying unit, The management server includes: A learning unit that performs learning based on the learning data received from the diagnostic cloud server; a function update data generation unit that outputs a learning result by the learning unit as function update data for the edge device.

11. 11. The distributed system according to claim 10, A distributed system in which the function update data generated by the management server is input to the recognition unit, the judgment unit, or the unexpected event judgment unit via the communication unit.

12. 10. The distributed system according to claim 9, The unexpected event includes not only areas that may lead to a collision, but also areas that do not lead to a collision and that exceed the safety assessment boundary, in a distributed system.

Images