Vehicle assembly system and method of assembling a vehicle

Abstract

A vehicle assembly system and a method of assembling a vehicle are disclosed, wherein the vehicle assembly system comprises: a vehicle chassis of a vehicle; at least one object sensor mounted to the vehicle chassis, wherein the at least one object sensor generates sensor data based on at least one detected object; a vehicle controller mounted to the vehicle chassis and configured to receive the sensor data from the at least one object sensor, wherein during assembly, the vehicle controller is configured with production control software that enables the vehicle controller to: generate production object data from the sensor data; monitor for a safety event based on the production object data; and generate a safety event signal in response to detecting a full event; and a safety controller configured to: receive the safety event signal from the vehicle controller; and change movement of a monitored machine corresponding to the safety event.

Description

TECHNICAL FIELD

[0001] The present disclosure generally relates to a sensor system and a method for monitoring an environment during manufacturing using the sensor system. BACKGROUND

[0002] Today, vehicle production and end drive applications are considered as completely separated fields. Vehicle production refers to vehicle manufacturing or vehicle repair. In contrast, end drive applications refer to operating a vehicle (e.g., driving a vehicle) outside of vehicle production.

[0003] In vehicle manufacturing, only the chassis construction is fully automated, whereas vehicle assembly is still a highly manual labor-intensive task and is only tool-assisted to some extent. In order to further increase the level of automation, there is a high demand for more intensive deployment of robotic systems as part of the vehicle assembly process.

[0004] In contrast to chassis construction, there is a higher degree of flexibility for assembly set-ups for vehicles. This requires a collaborative set-up between human workers and intelligent machines. The demand for flexibility and collaboration enforces an open work space in which humans and machines are no longer separated by a protective barrier.

[0005] Therefore, so-called safety Light Detection and Ranging (LIDAR) scanners have become a possible solution for protecting humans working within the same area in which machines are operating. These safety LIDAR scanners are arranged outside of the vehicle, around the perimeter of the work station (i.e., production cell) and face inwardly towards the vehicle chassis. Thus, the safety LIDAR scanners cover some of the areas in which humans and machines are operating.

[0006] The automation of complex assembly tasks not only requires intelligent safety solutions for collaborative set-ups, but also powerful solutions for guidance of robots and machines in highly complex three-dimensional (3D) work environments. Solutions for the following are required: recognition, positioning and guidance of different machine parts (e.g., articulated robot arms) to different chassis target positions; recognition and positioning of parts to be installed within the vehicle; and guidance for picking up these parts (e.g., via articulated robot arms) for installation to the vehicle in the assembly.

[0007] A commonality of all scanning solutions or mirror solutions for machine safety and functional guidance is the fact that the respective sensors have a fixed set-up at the production cell or directly at the respective machine (e.g., end effector tool of a robot). Thus, the sensors are positioned outside of the vehicle and away from the vehicle and typically scan inwardly into the work environment in which humans can be working.

[0008] In contrast to this manufacturing-centric scanning setup for automated vehicle production, there is an increasing number of separate scanning devices that will be installed in and around the vehicle itself. Using these new scanning sensors (e.g. radar, lidar, ultrasound, RGB cameras, time-of-flight cameras, 3D cameras, etc.) to collect information about the vehicle's close and medium range environment for driving assistance (e.g. lane assist, parking assist, adaptive cruise control, traffic sign detection, etc.) and active (passenger) safety systems. These scanning sensors scan the vehicle's exterior environment during post-production (e.g. during operation). That is, these scanning sensors scan outward from the vehicle.

[0009] Additionally, to gain better monitoring capabilities inside the vehicle, another set of scanning sensors is also installed inside the vehicle. That is, these scanning sensors scan the vehicle's interior environment during post-production. For example, next generation airbag systems can use sensors to determine the positioning and location of each passenger positioned inside the vehicle. This becomes even more important for autonomous driving vehicles, as people choose seating positions differently than is common in manually driven vehicles. Additionally, to support algorithms for early fatigue detection, anti-theft, etc., face scanners are being introduced. Sensor-wise, time-of-flight 3D cameras, radar sensors, etc. can also be used.

[0010] The production sensors for vehicle production and the vehicle sensors for vehicle operation are completely decoupled from each other. Typically, the sensors for vehicle operation are not activated until vehicle production is complete. Similarly, the algorithmic engines coupled to the production sensors and the algorithmic engines coupled to the vehicle sensors are completely decoupled and strictly separated with respect to their functional focus. The sensors and scanners on the vehicle itself only enter actual functional use after the vehicle has completed the final manufacturing stages and is ready to be put into traffic. In contrast, the production line needs to be equipped with thousands of additional scanning sensors and systems that enable automated and safe vehicle production.

[0011] Therefore, there is an increasing amount of redundancy that stems from the fact that vehicle sensors and production sensors are considered separate systems, while both need to adhere to functional safety standards. This incurs additional costs within the production line. The costs can be realized both in the cost for each safety scanner used in the production line, but also in wasted production space for safety scanners and more complex protective setups for the production units.

[0012] Therefore, an improved system that employs a "dual-purpose" vehicle-mounted (scanning) sensor for both vehicle production and vehicle operation can be desirable. SUMMARY

[0013] One or more embodiments provide a vehicle assembly system comprising: a vehicle chassis of a vehicle; at least one object sensor mounted to the vehicle chassis, wherein the at least one object sensor generates sensor data based on at least one detected object; a vehicle controller mounted to the vehicle chassis and configured to receive the sensor data from the at least one object sensor, wherein during assembly, the vehicle controller is configured with production control software that enables the vehicle controller to: generate production object data from the sensor data; monitor for a safety event based on the production object data; and generate a safety event signal in response to detecting the safety event; and a safety controller configured to: receive the safety event signal from the vehicle controller; and change movement of a monitored machine corresponding to the safety event.

[0014] One or more embodiments provide a method of assembling a vehicle. The method comprises: mounting at least one object sensor to a vehicle chassis of the vehicle; mounting a vehicle controller to the vehicle chassis; generating, by the at least one object sensor, sensor data based on at least one object detected by the at least one object sensor; installing, in a memory of the vehicle controller, production control software; generating, by the vehicle controller based on the production control software, production object data from the sensor data; monitoring, by the vehicle controller based on the production control software, for a safety event based on the production object data; generating, by the vehicle controller based on the production control software, a safety event signal in response to detecting the safety event; and changing, by a safety controller based on the safety event signal, movement of a monitored machine corresponding to the safety event. BRIEF DESCRIPTION OF DRAWINGS

[0015] Embodiments are described herein with reference to the accompanying drawings.

[0016] Figures 1A-1C A dual use scanning system according to one or more embodiments is shown;

[0017] Figure 2 A dual use scanning system according to one or more embodiments is shown; and

[0018] Figure 3A and Figure 3B A vehicle chassis of a vehicle in assembly at two stages of production according to one or more embodiments is shown. DETAILED DESCRIPTION

[0019] In the following, various embodiments will be described in detail with reference to the attached drawings. It should be noted that these embodiments are only for illustrative purposes and should not be construed as limiting. For example, although the embodiments can be described as including a plurality of features or elements, this should not be construed as indicating that all of these features or elements are required in order to implement the embodiments. Rather, in other embodiments, some of the features or elements can be omitted, or replaced by alternative features or elements. In other instances, well-known structures and devices are shown in block diagram form or in schematic form rather than in detailed form in order to avoid obscuring the embodiments.

[0020] Furthermore, in the following description, equivalent or similar elements or elements having equivalent or similar functions are denoted by equivalent or similar reference numerals. Since the same elements or functionally equivalent elements are given the same reference numerals in the drawings, repeated description of the elements provided with the same reference numerals can be omitted. Thus, the description provided for the elements having the same or similar reference numerals is exchangeable.

[0021] Features from different embodiments can be combined to form additional embodiments unless specifically stated otherwise. Changes or modifications to the described embodiments can also be made without departing from the scope of the claims. In some instances, well-known structures and devices are shown in block diagram form or in schematic form rather than in detailed form in order to avoid obscuring the embodiments.

[0022] Unless otherwise indicated, connections or couplings between elements shown in the drawings or described herein can be a wired connection or a wireless connection. Furthermore, such connections or couplings can be a direct connection or coupling without additional intervening elements, or an indirect connection or coupling with one or more additional intervening elements, as long as the general purpose of the connection or coupling (e.g., for transmitting a certain signal or transmitting a certain information) is maintained in essence.

[0023] Embodiments relate to sensors and sensor systems for obtaining sensor data about an environment. A sensor can refer to a component that converts a physical quantity to be measured into an electrical signal, e.g., a current signal or a voltage signal. More specifically, embodiments relate to object sensors comprising a pressure sensor, a contact switch (e.g., mounted inside a passenger seat), and a scanning sensor that detects and / or measures magnetic or electromagnetic radiation (e.g., radio waves, microwaves, infrared light, and visible light) as a physical quantity in order to generate sensor data representative of a field of view (FOV) of the respective scanning sensor. Scanning sensors can include, but are not limited to, 3D magnetic sensors, radar sensors, Light Detection and Ranging (LIDAR) sensors, time-of-flight 3D cameras, ultrasonic sensors, RGB cameras, Charge-Coupled Device (CCD) sensors, etc. Thus, some scanning sensors can also be referred to as imaging sensors.

[0024] Additionally, the one or more processors can be configured to receive sensor data and convert the sensor data into object data and / or image data via signal processing. For example, the one or more processors can use sensor data generated by the one or more scanning sensors to generate two-dimensional (2D) information and / or three-dimensional (3D) information. The 2D information and 3D information can be object data, e.g., point cloud data, and the one or more processors can be configured to detect and / or monitor objects in the FOV based on the sensor data generated by the one or more scanning sensors. In this way, each scanning sensor can be used to monitor, detect, classify, and track objects in its FOV. Additionally, other types of sensors, e.g., pressure sensors and contact switches, can be used for object detection. Thus, the one or more processors can receive sensor data from one or more of these sensors to detect objects and generate object data.

[0025] Each object sensor can be arranged to monitor a corresponding area inside or outside the vehicle. For example, a pressure sensor or contact switch can be installed inside a passenger seat to monitor objects that will come into contact with the seat. Additionally, scanning sensors can be arranged to monitor different sub-areas, e.g., different FOVs. A central processor can be used to receive sensor data from each scanning sensor and monitor, detect, classify, and track objects in each sub-area or FOV. The central processor can be integrated with a system controller that controls the system based on information generated by the central processor, i.e., based on the sensor data and detected objects in each FOV. The system can be, for example, a vehicle assembly system used in vehicle manufacturing. Thus, the system controller can include at least one processor and / or at least one processor circuit (e.g., a comparator and a digital signal processor (DSP)) of a signal processing chain for processing the sensor data, and a control circuit, e.g., a microcontroller, configured to generate control signals. Additionally, a trained artificial neural network (aNN), i.e., one or more processors programmed for aNN analysis, can also be used to analyze the sensor data and evaluate valid and invalid position patterns and valid and invalid motion patterns.

[0026] Figures 1A-1C A dual-use detection system 100 according to one or more embodiments is shown. In particular, Figure 1A and Figure 1B Two sets of external object sensors (e.g., scanning sensors) are shown for monitoring the external environment of an assembled vehicle 10. The assembled vehicle 10 includes at least a vehicle chassis and can include additional vehicle parts and components that are added to the vehicle chassis during production. Figure 1AOne set of external object sensors shown in FIG. 1 includes production sensors 1 arranged around a perimeter 11 of the production cell 12 (i.e., around a perimeter of the vehicle under assembly 10) and scanning inwardly from the perimeter 11 of the production cell 12 toward the vehicle under assembly 10. Figure 1B Another set of external object sensors shown in FIG. 1 includes vehicle sensors 2 mounted on the vehicle under assembly 10 (e.g., radar sensors, light detection and ranging (LIDAR) sensors, time-of-flight 3D cameras, ultrasonic sensors, RGB cameras, charge-coupled device (CCD) sensors, etc.). The vehicle sensors 2 scan outwardly from the vehicle under assembly 10 into the external environment.

[0027] Figure 1C One set of internal object sensors is shown including vehicle sensors 3 mounted on the vehicle under assembly 10 (e.g., pressure sensors, contact switches, 3D magnetic sensors, radar sensors, light detection and ranging (LIDAR) sensors, time-of-flight 3D cameras, ultrasonic sensors, RGB cameras, charge-coupled device (CCD) sensors, etc.) and used to monitor the internal environment (e.g., passenger cabin) of the vehicle under assembly 10. Together, the vehicle sensors 2 and the vehicle sensors 3 constitute a network of vehicle sensors mounted directly or indirectly to the vehicle chassis.

[0028] The vehicle under assembly 10 also includes at least one vehicle controller 13 configured to receive sensor data generated by each of the mounted vehicle sensors 2 and vehicle sensors 3. The vehicle controller 13 can also be referred to as a safety controller.

[0029] For example, the vehicle controller 13 can be an electronic control unit (ECU) mounted within the vehicle under assembly 10. An ECU is any embedded system in automotive vehicle electronics that controls one or more of the electrical systems or subsystems in a vehicle. An ECU can include a microcontroller, memory, embedded software stored in the memory, input devices, output devices, and a communication link.

[0030] It is noted that when the production sensor 1 is fixed to a specific production cell 12, the installed in-vehicle sensors 2 and 3 move along the production line from production cell to production cell with the vehicle 10 under assembly. In addition, not all vehicle sensors 2 and 3 can be initially installed, but some vehicle sensors 2 and 3 can be installed at different stages of vehicle assembly. Once the dual-purpose vehicle sensors 2 and / or 3 are installed, they will be permanently installed in the vehicle and they can be activated and used for object detection and tracking during the remainder of vehicle assembly. Further, it will be appreciated that the production sensor 1 can be optional during one or more production stages. Here, object detection can rely solely on the vehicle sensors 2 and 3.

[0031] As will be described in more detail, the vehicle sensors 2 and 3 are used as “dual-purpose” for both vehicle production as well as later vehicle product during vehicle operation. This requires loading two types of control software into the vehicle controller 13. The control software can also refer to aNN programs that are differently trained for production and driving applications.

[0032] For example, during an initial stage of production, the vehicle controller 13 can be installed into the vehicle 10 under assembly with the production control software 14 loaded into the memory 17. Alternatively, the production control software 14 can be installed into the vehicle controller 13 shortly after installation of the vehicle controller 13. The production control software 14 is used during production of the vehicle 10 under assembly and interacts with sensor data generated by the vehicle sensors 2 and 3 during production. In particular, the production control software 14 is used by the vehicle controller 13 for generating production object data from the sensor data and analyzing the production object data for safety events involving production-specific object violations and production area violations. The production object data can include human objects, vehicle objects (e.g., expected vehicle components), and production objects (e.g., production equipment such as robots). The vehicle controller 13 is configured to actively monitor and detect safety events during production of the vehicle based on production object data derived from sensor data received from the vehicle sensors 2 and 3.

[0033] Subsequently, during the final stages of production, after the vehicle assembly is complete or substantially complete, the memory 17 of the vehicle controller 13 is re-flashed with drive control software 15 for operation of the vehicle. The drive control software 15 is used during operation of the vehicle and interacts with sensor data generated by the vehicle sensors 2 and 3 during the post-production period, i.e. during operation of the vehicle. It will also be appreciated that the memory 17 of the vehicle controller 13 can be re-flashed again with service control software used in a similar manner to the production control software 14, instead of direct service of the vehicle to a service station or repair shop. After the service is complete, the memory 17 of the vehicle controller 13 can be re-flashed again with the drive control software 15.

[0034] Alternatively, the program for use at the repair shop can be part of the software stack released when the vehicle goes out of production. In other words, the drive control software 15 can include additional programs so that the vehicle sensors 2 and 3 can be used again to detect safety events (similar to the detection used during production), but this time in conjunction with vehicle service.

[0035] Accordingly, the described embodiments present the technical concept of a "dual use" vehicle on-board object sensor that includes a setup for protecting or simplifying the vehicle production process by benefiting from on-board sensors and scanners that are better suited for near target scanning. Accordingly, each on-board sensor and scanner can be used both in production and during operation of the final product.

[0036] Figure 2 A dual use detection system 200 according to one or more embodiments is shown. The dual use detection system 200 is similar to the dual use detection system 100, but the dual use detection system 200 includes a production control system for controlling different machines of the production line. For example, the production control system can control articulated robotic arms and other robotic mechanisms as well as production line mechanisms such as conveyors. The production control system includes a safety programmable logic controller (PLC) 21, a robot controller 22, and a standard process automation control system 23. Also shown are the production control software 14 and the drive control software 15 to be loaded into the memory 17 of the vehicle controller 13 as described above. The production control software 14 and the drive control software 15 can be installed by a software controller 16 such as a computer that is communicatively coupled to the vehicle controller 13. The software controller 16 can also be used to reprogram or reconfigure the production control software 14 as will be described in more detail below.

[0037] The safety PLC 21 is coupled to the vehicle controller 13 via a fieldbus 24 for receiving sensor data, object data, positioning data, warning signals, and / or control signals from the vehicle controller 13 in real-time. While not limited thereto, the fieldbus 24 can use OPC Unified Architecture (OPC UA), Ethernet Time-Sensitive Networking (TSN), or a secure and reliable wireless technology for communication between the safety PLC 21 and the vehicle controller 13. Thus, the fieldbus 24 represents a communication channel.

[0038] The vehicle controller 13 receives sensor data from the vehicle sensors 2 and 3 and can generate object data, positioning data, warning signals, and / or control signals based on the sensor data applied to the production control software 14. The production control software 14 includes object presence detection algorithms and object positioning algorithms adapted to the production of the vehicle 10 under assembly. These object presence detection algorithms and object positioning algorithms are configured to detect, identify, position, and track objects during production using the sensor data. The production control software 14 is configured to generate object data including object identification and positioning data from the sensor data and use the object data to detect, for example, safety events. The object data can also be output from the vehicle controller 13 for use by the robot controller 22 and the standard process automation control system 23 to assist the production guidance system.

[0039] A safety event (e.g., a safety zone violation) is a potentially dangerous or imminent dangerous situation that involves a human worker or other object located within the production cell, where a monitored machine also operating within the production cell can cause physical harm to the human worker or damage to the other object if the safety PLC 21 does not intervene.

[0040] Upon detecting a safety event, the vehicle controller 13 can generate a safety event signal, e.g., a warning signal or a control signal, and send the safety event signal to the safety PLC 21 via the fieldbus 24 or other communication channel. The warning signal can identify a warning type or threat level corresponding to the safety event, and the safety PLC 21 can select one of a plurality of actions to take in response to the identified warning type or threat level. For example, the vehicle controller 13 can be configured to assess a threat level based on an object type (e.g., human or non-human) and proximity of the object to a perceived threat, where a human object is in close proximity (e.g., less than a predetermined threshold distance) to a perceived threat that warrants some action.

[0041] At least two types of safety events can be identified, including a non-critical zone violation and a critical zone violation. A non-critical zone violation occurs when a human is within a first predetermined threshold distance of a monitored robot, despite the monitored robot moving away from the human. In this first case, the monitored robot can slow down. In contrast, a critical zone violation occurs when a human enters a protected robot space and / or is within a first predetermined threshold distance of a monitored robot and the monitored robot moves toward the human. In this second case, the threat level is higher than in the first case, and the monitored robot can be stopped. It will also be appreciated that different distances can be used for different threat levels, including those described above.

[0042] Another zone violation can occur when a human is within a second predetermined threshold distance of a monitored robot but is further from the monitored robot than the first predetermined threshold distance and the monitored robot moves toward the human. In this third case, the threat level is lower than in the above second case, and the monitored robot can slow down. The vehicle controller 13 continues to monitor for critical zone violations.

[0043] Alternatively, the vehicle controller 13 can select one of a plurality of actions to take based on the safety event, and command the safety PLC 21 to take the selected action via a control signal. Both the warning signal and the control signal can be accompanied by object data identifying the monitored machine corresponding to the safety event for which the safety PLC 21 should act.

[0044] Upon receiving the warning signal or the control signal from the vehicle controller 13, the safety PLC 21 can perform one or more safety measures including taking control of the monitored machine corresponding to the safety event to prevent injury to the human worker or damage to the object. Control of the monitored machine can include triggering a slowdown of the operating speed of the monitored machine, or even triggering a full“safety stop” of the monitored machine, depending on the level of penetration and the threat level assessed to the human worker or object. If the safety PLC 21 triggers a safety stop, the safety PLC 21 can hold the position of the monitored machine for at least a predetermined amount of time. Additionally, the safety PLC 21 can require a manual restart in order to restart production after the safety event.

[0045] Additionally or alternatively, the vehicle controller 13 can be configured as an interface to receive sensor data from the vehicle sensors 2 and 3 and to forward the sensor data seamlessly to the safety PLC 21 for object detection, identification, localization, and tracking. In this case, the safety PLC 21 is configured to receive sensor data from the vehicle sensors 2 and 3 in real-time via the vehicle controller 13 for generating object data. Based on this sensor data, the safety PLC 21 detects, classifies, localizes, and tracks objects within the production unit, including the identification of humans and machines, and detects safety events to occur (e.g., safety zone violations) based on the detected objects. Thus, additionally or alternatively, any of the functions described herein as being performed by the vehicle controller 13 can be performed by the safety PLC 21 based on received sensor data and / or object data. Additionally, the PLC 21 can also be configured to assess a threat level to human workers or objects and take one of a plurality of measures based on the determined threat level.

[0046] A hybrid setup is also possible, in which both the vehicle controller 13 and the safety PLC 21 receive sensor data from the vehicle sensors 2 and 3 in order to detect, classify, localize, and track objects within the production unit.

[0047] As a result of using the vehicle sensors 2 and 3 for object detection during production, the vehicle controller 13 and at least one external vehicle sensor 2 and / or internal vehicle sensor 3 should be installed during an initial phase of the production sequence, with the possibility of adding additional vehicle sensors 2 and 3 at a later stage of the production sequence.

[0048] Additionally, the safety PLC 21 can optionally be coupled to one or more production sensors 1 via the field bus 25 for receiving sensor data generated by the production sensors 1. While not limited thereto, the field bus 25 can use OPC UA technology, Ethernet TSN technology, or wireless technology for communicating between the safety PLC 21 and the production sensors 1. Thus, the field bus 25 represents a communication channel. Here, the safety PLC 21 can be configured to generate additional object data based on sensor data received from the production sensors 1 and use this additional object data to detect safety events.

[0049] The robot controller 22 is configured to control the robotic mechanisms used in the production line and more specifically in one or more production cells. The robot controller 22 differs from the safety PLC 21 in that the robot controller 22 is configured to control the robotic mechanisms based on programmed assembly control software according to an assembly sequence of the vehicle 10 in assembly. Thus, vehicle components are mounted on or within the vehicle 10 in assembly via the robot controller 22.

[0050] The robot controller 22 is optionally coupled to the vehicle controller 13 for receiving sensor data or object data from the vehicle controller 13. The sensor data can be used by the robot controller 22 to generate object data to assist vehicle assembly. For example, the robot controller 22 can detect, classify, and localize objects to be moved by articulated robotic arms. Thus, the sensor data can be used by the robot controller 22 for tool steering, object localization, and for guiding movements of the robotic mechanisms to one or more objects detected with the sensor data. Alternatively, the robot controller 22 can receive object data from the vehicle controller 13 that can be used to control movements of at least one monitored machine.

[0051] In general, the vehicle controller 13 is configured with production control software 14 that enables the vehicle controller 13 to: generate production object data from sensor data; monitor a position of at least one monitored machine (e.g., at least one production machine such as a robotic arm) based on the production object data; and generate position information based on the position of the at least one monitored machine. The robot controller 22 can receive the position information and control movements of the at least one monitored machine based on the position information. Alternatively, the robot controller 22 can generate position information for machine control based on object data received from the vehicle controller 13. Thus, the robot controller 22 can be used for non-safety related control such as tool steering, object localization, and quality checks during production. The robot controller 22 can operate in this manner with or without the safety PLC 21. In other words, the embodiments are not limited to safety aspects and can be used with or without the safety PLC 21.

[0052] For similar purposes, the robot controller 22 is optionally coupled to the production sensors 1. The sensor data can be used by the robot controller 22 to generate object data to assist vehicle assembly including guiding movements of the robotic mechanisms to one or more objects detected with the sensor data.

[0053] The robot controller 22 is also communicatively coupled to the safety PLC 21 for receiving control signals from the safety PLC 21. The safety PLC 21 can generate a control signal in response to detecting or receiving a notification of a safety event and send the control signal to the robot controller 22 to control the robot controller 22. For example, the control signal can instruct the robot controller 22 to slow down or stop a monitored machine corresponding to the detected safety event.

[0054] The standard process automation control system 23 is a production line controller configured to control production line mechanisms, such as conveyors or production tracks, including transport automation guided vehicles (AGVs) and autonomous mobile robots (AMRs). The standard process automation control system 23 differs from the safety PLC 21 in that the standard process automation control system 23 is configured to control the production line mechanisms based on a programmed assembly control software according to an assembly sequence of the vehicle 10 under assembly. Thus, the vehicle 10 under assembly is moved along the production line and advanced from one production stage to the next production stage via the standard process automation control system 23.

[0055] The standard process automation control system 23 is optionally coupled to the vehicle controller 13 for receiving sensor data or object data from the vehicle controller 13. The sensor data can be used by the standard process automation control system 23 to generate object data to assist with vehicle assembly. For example, the standard process automation control system 23 can detect and localize the vehicle 10 under assembly and its components in order to adjust the speed of the production line mechanisms. For example, the standard process automation control system 23 can detect whether a production stage is completed and advance the vehicle 10 under assembly to the next production stage if the production stage is completed. Alternatively, the object data received from the vehicle controller 13 can be used for this purpose. Thus, the standard process automation control system 23 can be used for non-safety related control of the production line mechanisms, such as for object localization and quality checks during production. The standard process automation control system 23 can operate in this way with or without the safety PLC 21 for safety aspects. In other words, the embodiments are not limited to safety aspects and can be used with or without the safety PLC 21.

[0056] For similar purposes, the standard process automation control system 23 is optionally coupled to the production sensors 1. The sensor data can be used by the standard process automation control system 23 to generate object data to assist with vehicle assembly, including guiding and adjusting the production line mechanisms.

[0057] The standard process automation control system 23 is also communicatively coupled to the safety PLC 21 for receiving control signals from the safety PLC 21. The safety PLC 21 can generate a control signal in response to detecting a safety event or receiving a notification of a safety event, and send the control signal to the standard process automation control system 23 to control the standard process automation control system 23. For example, the control signal can instruct the standard process automation control system 23 to slow down or stop the production line mechanisms of the production unit corresponding to the detected safety event.

[0058] Based on the dual-purpose detection system 200, the built-in sensors 3 for monitoring the interior of the vehicle (e.g., for passenger detection and / or scanning in vehicle operation) and the exterior sensors 2 for exterior environment scanning (e.g., for driving assistance in vehicle operation) are used for detection and scanning purposes during vehicle production. Thus, the number of separate and expensive detection equipment for production purposes can be reduced, and the observed field of view is improved. Since the vehicle sensors already exist in the vehicle, which are cost-effective, intelligent and have high performance, the vehicle sensors can help to reduce production costs while enhancing safety. Repetition of scanners / sensors between the vehicle-provided own settings and separate production line settings can be avoided. Furthermore, even at the time of assembly, the vehicle's own interior scanners are already located in the best possible area to ensure the most efficient field of view settings. Compared to trying to enable a collaborative work setting by scanning the infrastructure from the outside inside the vehicle, a better detection setting can be achieved with the vehicle's own interior sensors / scanners.

[0059] To realize such a production setting, data from various detection functions of the vehicle are made available to the external process and safety controllers 21 to 23 via wired or wireless communication infrastructure. The embedded algorithmic control of the various vehicle sensors 2 and 3 as well as scanners, i.e., the production control software 14 and the driving control software 15, needs to be programmed differently between production and driving operation. When the vehicle leaves the production line, a switch is made to the latest driving operation. Thus, the existing vehicle infrastructure, here the scanners as well as the sensors 2 and 3, is reused to simplify the production process.

[0060] To introduce the above concept into next generation vehicle production, one or more of the following changes can be implemented:

[0061] (1) The vehicle design for scanner / sensor settings can be modified according to current design, so that scanners and sensors are installed to the vehicle under assembly at an early stage of the production sequence, in order to enable the most efficient effect of using vehicle scanners and sensors for production;

[0062] (2) The vehicle and process control system can interface in a standard way, which does not interfere with the existing vehicle control and production automation control architecture (hardware and software);

[0063] (3) This new interface can be highly secure to avoid hacking that can violate the safety concept of the vehicle;

[0064] (4) The control algorithms can be adapted for production and traffic related aspects;

[0065] (5) A fast communication link and reprogramming infrastructure can be provided, which allows updating the vehicle scanner and sensor functionality at the end of assembly before the vehicle leaves the production line;

[0066] (6) By a combination of 3D scanning concepts of time-of-flight and radar, the conventional scanning concept of only 2D LIDAR can be replaced by a more powerful, sensor fusion oriented approach; and

[0067] (7) Machine learning algorithms for advanced object detection algorithms and embedded artificial neural network technology can be actively deployed in the intelligent sensors and intelligent scanners.

[0068] As described above, the vehicle scanner, as well as sensors 2 and 3, are interfaced in real time to the external safety PLC 21, the robot controller 22 and the standard process automation control system 23. The vehicle controller 13 is able to exchange its embedded sensor / scanner data in real time with the externally connected production control system 21 to 23 seamlessly using, for example, OCP-UA, Ethernet TSN, advanced wireless technology, etc.

[0069] During production, the object presence detection, object classification and object localization algorithms of the production control software 14 use the on-board scanner, as well as sensors 2 and 3, and the embedded edge controller. These object presence detection and object localization algorithms are configured to detect, identify, classify, localize and track objects during production using the sensor data. This object data can be used by the external safety PLC 21 during production to detect, for example, safety events, and can be used by the robot controller 22 and the standard process automation control system 23 as object information for the production guidance system.

[0070] During the final stages of production, after the vehicle is assembled, the vehicle controller 13 is reprogrammed or reconfigured in a quick and efficient manner using the drive control software 15 without violating various safety concepts required for production and traffic. The drive control software 15 includes object presence detection algorithms, object classification algorithms, and object localization algorithms configured for drive applications including, but not limited to, drive assist (e.g., lane keep assist, parking assist, adaptive cruise control, traffic sign detection, etc.) and active (passenger) safety systems. Thus, these object presence detection algorithms and object localization algorithms are configured to use sensor data during vehicle operation, post assembly, to detect, identify, classify, localize, and track objects.

[0071] Figure 3A and Figure 3B A vehicle chassis of the vehicle 10 in assembly at two stages of production is shown in accordance with one or more embodiments. In particular, Figure 3A A vehicle chassis of the vehicle 10 in assembly is shown, which is empty except for the vehicle controller 13 and one or more vehicle sensors 3. For simplicity, only one interior vehicle sensor 3 is shown. In contrast, Figure 3B A vehicle chassis of the vehicle 10 in assembly is shown at a subsequent stage of production, after the front seats 31 and steering wheel 32 are installed into the vehicle chassis.

[0072] To account for the different stages of production, the production control software 14 can be reprogrammed and reconfigured multiple times during production in accordance with the current assembly state of the vehicle 10 in assembly. As Figure 3A and Figure 3B As shown, the interior and exterior makeup of the vehicle 10 in assembly will change as components are added to the vehicle through various stages of assembly. Thus, the observation space monitored by the vehicle sensors 2 and vehicle sensors 3 frequently changes throughout the production process.

[0073] In addition, as additional vehicle sensors 2 and / or vehicle sensors 3 are installed to the vehicle 10 in assembly through various stages of production, the production control software 14 can be updated. Thus, the production control software 14 can be updated when the additional sensors 2 and / or additional sensors 3 are activated.

[0074] Accordingly, the vehicle controller 13 should be reconfigured by updated production control software 14 so that the vehicle controller 13 can distinguish between known expected objects and unexpected objects and critical objects (e.g., humans) to avoid triggering false safety events and better assist the production guidance system. Accordingly, the production control software 14 can be updated after a vehicle component is installed to the vehicle 10 under assembly to recognize the installed vehicle component as an expected object. Without such adaptation of the production control software 14, the vehicle controller 13 can report false alarms or incorrect localization data. Additionally, the production control software 14 can be updated after a sensor component is installed to the vehicle 10 under assembly to increase or enhance the area coverage of the sensor network.

[0075] The frequent adjustments to the vehicle sensor 2 and the vehicle sensor 3 can also be seen as modified task profiles for the vehicle sensor 2 and the vehicle sensor 3. Accordingly, the production control software 14 can include multiple task profiles, for example, assigned or mapped to one or more production phases. The vehicle controller 13 can receive production information from, for example, a standard process automation control system 23 that identifies the current production phase of the production line in which the vehicle under assembly resides. Using the production information, the vehicle controller 13 can selectively activate one of the task profiles and deactivate the other task profiles to bring the production control software 14 in line with the current production phase. The task profiles can take into account the current assembly state of the vehicle 10 under assembly, including installed vehicle components and the vehicle sensor 2 and the vehicle sensor 3. Different task profiles produce differences in supported fields of view, object detection, and alarm scenarios.

[0076] To provide better control in the various task profiles, object and detection classification can be performed by the production control software 14 using aNNs with machine learning and / or deep learning. Since the modification of the required scanning field of view is fully known according to the actual assembly phase of the vehicle, the aNNs will be taught how the environment changes and what the expected obstacles are and how they differ from the expected scanning setup, and thus what the system should pay attention to. Since low-cost aNN solutions are currently being developed, such aNN solutions can be directly paired with each respective sensing system, making the sensing system a smart sensor. Shaping the field of view according to the expected obstacle setup can be done as aNN training during initial production line start-up and can also account for vehicle variants, and further can be “fed” by large-scale cloud computing. Once the configuration parameters for the aNNs are optimized, they can be fed into the local aNNs, where they are also fixed to comply with safety standards without any further modifications.

[0077] The configuration parameters for an aNN are the weights of the computation of each neuron transfer function. The computation of these weights is performed during the training of the aNN and requires high computational power. Once the weights are retrieved, classification and prediction are performed.

[0078] In particular, pairing vision scanning technology with radar opens new technological opportunities for scanner set-ups. Previously, identifying objects or determining the presence of obstacles was based on either geometric analysis within a higher level algorithm layer (edge or basic shape detection, etc.) or simple (high resolution) reflectometry measurements. Radar also provides the additional opportunity to extract information about the material of the scanned object (e.g., metal vs. human body) to assist in object identification. This functionality can complement a combined scanner not only in terms of safety but also in terms of advanced object differentiation functionality by enabling combined material and object detection that can be used for quality checks during assembly during the production process. This can help reduce the number of false safety events or the number of undetected safety events by ensuring that human objects are quickly and correctly detected.

[0079] In light of the above, opening and expanding the vehicle’s built-in sensing and scanning solutions early in production to enable more advanced vehicle assembly would pave the way for higher levels of automation at attractive automation costs.

[0080] According to one or more embodiments, a vehicle assembly system includes: a vehicle chassis of a vehicle; at least one object sensor mounted to the vehicle chassis, wherein the at least one object sensor generates sensor data based on at least one detected object; at least one processor configured to: receive the sensor data from the at least one object sensor, wherein during assembly, the at least one processor is configured with production control software that enables the at least one processor to: generate production object data from the sensor data; monitor for a safety event based on the production object data; and generate a safety event signal in response to detecting a full event; and a safety controller configured to: receive the safety event signal from the at least one processor; and change movement of a monitored machine corresponding to the safety event. The at least one processor can be integrated in a vehicle controller 13 mounted to the vehicle chassis, can be integrated in the safety controller (i.e., safety PLC 21), or can be integrated in both the vehicle controller 13 and the safety controller, such that one or more actions are performed by the vehicle controller 13 and the remaining actions are performed by the safety controller.

[0081] According to one or more embodiments, a vehicle assembly system includes: a vehicle chassis of a vehicle; at least one object sensor mounted to the vehicle chassis, wherein the at least one object sensor generates sensor data based on at least one detected object; at least one processor configured to: receive the sensor data from the at least one object sensor, wherein during assembly, the at least one processor is configured with production control software that enables the at least one processor to: generate production object data from the sensor data; monitor a position of at least one monitored machine based on the production object data; and generate position information based on the position of the at least one monitored machine; and a controller configured to: receive the position information; and control movement of the at least one monitored machine based on the position information. The at least one processor can be integrated in a vehicle controller 13 mounted to the vehicle chassis, can be integrated in a safety controller (i.e., safety PLC 21), or can be integrated in both the vehicle controller 13 and the safety controller, such that one or more actions are performed by the vehicle controller 13 and the remaining actions are performed by the safety controller.

[0082] While various embodiments have been described, it will be apparent to one of ordinary skill in the art that other embodiments and implementations are possible that are within the scope of the disclosure. Additionally, it will be understood that the concepts described herein can be extended to non-automotive manufacturing, where sensors mounted to a manufacturer’s product are used during production for similar purposes as described herein. Accordingly, the present application is not to be limited, except as by the appended claims and equivalents thereof. With respect to the various functions that are described above as being performed by various components or structures (assemblies, devices, circuits, systems, etc.), unless otherwise specified, the terminology used to describe such components (including references to “means”) is intended to correspond to any component or structure that performs the specified function (i.e., that is functionally equivalent), even if not structurally equivalent to the disclosed structure that performs the function in the exemplary implementations of the application shown herein.

[0083] Furthermore, the appended claims are hereby incorporated into the detailed description, wherein each claim can stand as a separate example embodiment. While each claim can stand as a separate example embodiment, it is noted that while dependent claims can refer in the claims to a specific combination of one or more other claims, other example embodiments can include the combination of each other dependent claim or independent claim with the subject matter of the dependent claim. Such combinations are presented herein unless it is indicated that a specific combination is not intended. Furthermore, it is intended that features of the claims be included into any other independent claim, even if the claim is not directly incorporated by reference into the independent claim.

[0084] It should also be noted that the methods disclosed in the specification or in the claims can be implemented by a device having means for performing each of the respective actions of these methods.

[0085] Furthermore, it is to be understood that the disclosure of a number of actions or functions in the specification or in the claims can not be interpreted as an exclusive sequence of actions or functions. Thus, unless technically impossible, the disclosure of multiple actions or functions will not limit these to a particular order unless explicitly stated as such. Furthermore, in some embodiments, a single action can include or can be split into multiple sub-actions. Unless explicitly excluded, such sub-actions can be included within the disclosure of that single action and are part of the disclosure of that single action.

[0086] The techniques described in this disclosure can be implemented, at least in part, in hardware, software, firmware or any combination thereof. For example, various aspects of the described techniques can be implemented within one or more processors, including one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or any other equivalent integrated or discrete logic circuitry, as well as any combinations of such components. The term "processor" or "processing circuitry" can generally refer to any of the foregoing logic circuitry, alone or in combination, or any other equivalent electrical circuitry. A control unit comprising hardware can also perform one or more of the techniques of this disclosure. Such hardware, software and firmware can be

[0087] Further, a computing system or program, such as a trained aNN, can be implemented by one or more processors and / or control units. An aNN is an adaptive tool for nonlinear statistical data modeling that changes the aNN structure based on external or internal information flowing through the network during a learning phase. An aNN is a nonlinear statistical data modeling tool for modeling complex relationships between inputs and outputs or finding patterns in data.

[0088] While various example embodiments have been disclosed, it will be apparent to those of ordinary skill in the art that various changes and modifications can be made that will achieve some of the advantages described herein without departing from the spirit and scope of the invention. It will be obvious to those of ordinary skill in the art that other components performing the same functions can be suitably substituted. It should be understood that other implementations can be utilized and structural or logical changes can be made without departing from the scope of the present invention. It is intended that the appended claims and the legal equivalents thereof encompass any such changes and modifications.

Claims

1. A vehicle assembly system, comprising: The vehicle chassis of the vehicle to be assembled; At least one object sensor, said at least one object sensor mounted to the vehicle chassis, wherein said at least one object sensor generates sensor data based on at least one detected object; and At least one processor, configured to receive sensor data from the at least one object sensor, During the assembly of the vehicle, the at least one processor is configured with production control software that enables the at least one processor to: generate production object data based on the sensor data; identify monitored machines configured to assemble a portion of the vehicle based on the production object data during the vehicle assembly; monitor the position of the monitored machines based on the production object data; and monitor the monitored machines based on production object data corresponding to another detection object and the monitored position of the monitored machines in relation to the other detection object. The vehicle is assembled in multiple production stages, causing the vehicle's configuration to change with each of these stages. The production control software includes multiple profiles for detecting safety events related to the monitored machines. Each profile is mapped to a corresponding production stage among the multiple production stages, and each profile is associated with a different vehicle configuration and one or more monitored machines associated with that production stage. The at least one processor is configured to selectively activate and deactivate the plurality of profiles for detecting the security event based on the current production stage corresponding to the activated profile. The at least one processor is configured to monitor the safety events based on different vehicle configurations and one or more monitored machines associated with activated profiles mapped to the current production stage. The monitored machine is one of one or more monitored machines associated with the current production stage.

2. The vehicle assembly system according to claim 1, wherein, The at least one processor is configured to monitor the security event based on production object data corresponding to the other detection object and the monitored location of the monitored machine in relation to the other detection object, and to generate a security event signal in response to detecting the security event.

3. The vehicle assembly system according to claim 2, further comprising: A security controller configured to receive the security event signal from the at least one processor and to change the movement of the monitored machine corresponding to the security event.

4. The vehicle assembly system according to claim 1, wherein, The monitored machine is an articulated robotic arm.

5. The vehicle assembly system according to claim 1, wherein, The at least one object sensor includes at least one internal object sensor and at least one external object sensor, the at least one internal object sensor being configured to scan the internal environment of the vehicle chassis, and the at least one external object sensor being configured to scan the external environment of the vehicle chassis.

6. The vehicle assembly system according to claim 1, wherein: The at least one processor is configured to monitor security events related to the monitored machine based on production object data corresponding to the other detected object and the monitored location of the monitored machine in relation to the other detected object. The at least one processor is configured to detect whether the production object data includes human objects, and to detect the security event based on the at least one processor detecting that the human object is within a predetermined threshold distance from the monitored machine.

7. The vehicle assembly system according to claim 6, wherein: The at least one processor is configured to determine the threat level of the security event from multiple threat levels based on the production object data, and to generate a security event signal in response to detecting the security event. The security event signal includes threat level information corresponding to the determined threat level.

8. The vehicle assembly system according to claim 7, further comprising: A security controller configured to receive the security event signal from the at least one processor and to modify the movement of the monitored machine corresponding to the security event. The security controller is further configured to select one of a plurality of actions to alter the movement of the monitored machine based on the threat level information.

9. The vehicle assembly system according to claim 8, wherein, The multiple threat levels include: A first threat level, the first threat level being based on the at least one processor detecting that the human object is identified within a predetermined threshold distance from the monitored machine while the monitored machine is moving away from the human object; and The second threat level is based on the at least one processor detecting that the human object is within the predetermined threshold distance from the monitored machine while the monitored machine is moving toward the human object.

10. The vehicle assembly system according to claim 9, wherein: In response to threat level information indicating the first threat level, the security controller is configured to slow down the movement of the monitored machine, and In response to threat level information indicating the second threat level, the security controller is configured to stop the movement of the monitored machine.

11. The vehicle assembly system according to claim 1, further comprising: A production controller configured to track the current production stage of the vehicle; And sending production stage information to the at least one processor, the production stage information indicating the current production stage, The at least one processor is configured to selectively activate one of the plurality of profiles based on the current production stage indicated by the production stage information.

12. The vehicle assembly system according to claim 1, wherein: Each of the plurality of profiles includes a different set of intended objects to be mounted to the vehicle chassis, the set defining the vehicle configuration, and The at least one processor is configured to distinguish between unexpected and expected objects based on an activated profile used to detect the security event.

13. The vehicle assembly system according to claim 12, wherein, Detecting the expected object avoids triggering erroneous security events.

14. The vehicle assembly system according to claim 1, wherein, The vehicle assembly system also includes: A software controller configured to update production control software based on the current production stage of the plurality of production stages, wherein the updated production control software enables the at least one processor to monitor security events related to a monitored machine present in the current production stage, wherein the security events to be monitored are adapted to the current production stage based on a set of expected objects constituting the vehicle and mounted on the vehicle chassis.

15. The vehicle assembly system according to claim 14, wherein, The at least one processor is configured to distinguish between unexpected and expected objects based on an activated profile used to detect the security event.

16. The vehicle assembly system according to claim 1, further comprising: A vehicle controller, which is mounted to the vehicle chassis, wherein the vehicle controller includes the at least one processor.

17. The vehicle assembly system of claim 16, further comprising: A software controller configured to rewrite the production control software by refreshing the at least one processor using driving control software, wherein the driving control software enables the at least one processor to generate driving object data based on the sensor data and apply the driving object data to at least one driving function during driving operations of the vehicle.

18. The vehicle assembly system according to claim 17, wherein, The software controller is configured to refresh the at least one processor using the driving control software during the final stage of vehicle assembly.

19. The vehicle assembly system according to claim 17, wherein, The at least one driving function includes at least one of vehicle lane assist, vehicle parking assist, adaptive cruise control, traffic sign detection, and passenger detection.

20. The vehicle assembly system according to claim 1, further comprising: A controller configured to control the movement of the monitored machine based on the monitored location of the monitored machine in relation to the other detected object.

21. A method for assembling a vehicle, comprising: At least one object sensor is mounted to the chassis of the vehicle to be assembled; Sensor data is generated by the at least one object sensor based on at least one object detected by the at least one object sensor; Production control software is installed in the memory of at least one processor, wherein the production control software is configured for use during the assembly of the vehicle; The at least one processor generates production object data based on the sensor data using the production control software. The at least one processor identifies a monitored machine based on the production control software, the monitored machine being configured to assemble a portion of the vehicle based on the production object data during the vehicle assembly process. The at least one processor monitors the location of the monitored machine based on the production control software and the production object data. The at least one processor monitors the monitored machine based on the production control software, based on production object data corresponding to another monitored object and the monitored location of the monitored machine in relation to the other monitored object. The vehicle is assembled in multiple production stages, causing the vehicle's configuration to change with each of these stages. The production control software includes multiple profiles for detecting safety events related to the monitored machines, each profile being mapped to a corresponding production stage among the multiple production stages, and each profile being associated with a different vehicle configuration associated with the corresponding production stage and one or more monitored machines associated with the corresponding production stage. The at least one processor, based on the production control software, selectively activates and deactivates the plurality of profiles for detecting the security event based on the current production stage corresponding to the activated profile; and The safety events are monitored by at least one processor based on the production control software, based on different vehicle configurations and one or more monitored machines associated with activated profiles mapped to the current production stage. The monitored machine is one of one or more monitored machines associated with the current production stage.

22. The method of claim 21, further comprising: The at least one processor monitors the safety event based on the production control software, based on production object data corresponding to the other detection object and the monitored location of the monitored machine in relation to the other detection object. The at least one processor generates a security event signal in response to the detection of the security event, based on the production control software. as well as The safety controller modifies the movement of the monitored machine corresponding to the safety event based on the safety event signal.

23. The method according to claim 21, wherein, The monitored machine is an articulated robotic arm.

24. The method according to claim 21, wherein, The at least one object sensor includes at least one internal object sensor and at least one external object sensor, the at least one internal object sensor being configured to scan the internal environment of the vehicle chassis, and the at least one external object sensor being configured to scan the external environment of the vehicle chassis.

25. The method according to claim 22, wherein, Monitoring the security events includes: The monitored machine is detected based on the production object data; Detect whether the production object data includes human objects; and The security event is detected based on the fact that the at least one processor detects that the human object is within a predetermined threshold distance from the monitored machine.

26. The method of claim 25, wherein: Detecting the security event includes determining the threat level of the security event from multiple threat levels based on the production object data, wherein the security event signal includes threat level information corresponding to the determined threat level, and Changing the movement of the monitored machine corresponding to the security event includes selecting one of a plurality of actions for changing the movement of the monitored machine based on the threat level information.

27. The method according to claim 26, wherein, The multiple threat levels include: A first threat level, the first threat level being based on the detection that the human object is within a predetermined threshold distance from the monitored machine while the monitored machine is moving away from the human object; and The second threat level is identified based on detecting that the human object is within a predetermined threshold distance from the monitored machine while the monitored machine is moving toward the human object.

28. The method according to claim 27, wherein, Changing the movement of the monitored machine corresponding to the security event includes: In response to threat level information indicating the first threat level, the movement of the monitored machine is slowed down, and In response to threat level information indicating the second threat level, the movement of the monitored machine is stopped.

29. The method of claim 21, further comprising: The production controller tracks the current production stage of the vehicle; as well as The production controller sends production stage information to the at least one processor, the production stage information indicating the current production stage.

30. The method of claim 21, further comprising: The at least one processor monitors the safety event based on the production control software, based on production object data corresponding to the other detection object and the monitored location of the monitored machine in relation to the other detection object. The production control software is updated by the software controller based on the current production stage among the plurality of production stages, wherein the updated production control software enables the at least one processor to monitor safety events related to the monitored machine present in the current production stage, wherein the safety events to be monitored are adapted to the current production stage based on a set of expected objects constituting the vehicle and mounted on the vehicle chassis.

31. The method according to claim 30, wherein, The at least one processor is configured to distinguish between unexpected and expected objects based on an activated profile used to detect the security event.

32. The method of claim 21, further comprising: The software controller rewrites the production control software by refreshing the at least one processor using driving control software, wherein the driving control software enables the at least one processor to generate driving object data based on the sensor data and apply the driving object data to at least one driving function during the driving operation of the vehicle.

33. The method according to claim 32, wherein, The at least one driving function includes at least one of vehicle lane assist, vehicle parking assist, adaptive cruise control, traffic sign detection, and passenger detection.

Images