App quality confirmation system, app distribution device and app quality confirmation program

Abstract

An application quality confirmation system (1) comprises: a comparison unit (6a) that compares an operation signal of an application distributed from an application distribution device (3) to an on-vehicle machine (2), with at least one of a driver operation or a detection result from an autonomous sensor mounted on a vehicle; a normal operation determination unit (6b) that determines whether or not the application is operating normally on the basis of the comparison result of the comparison unit; a usage feasibility determination unit (11b) that determines, on the basis of the determination result of the normal operation determination unit, the feasibility of usage of the application; and a usage control unit (11c) that permits usage of the application when usage of the application is determined as feasible.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is based on Japanese Application No. 2022-202157, filed on December 19, 2022, the contents of which are incorporated herein by reference. [Technical Field]

[0002] The present disclosure relates to an application quality confirmation system, an application distribution device, and an application quality confirmation program. [Background technology]

[0003] In recent years, various applications that contribute to advanced driver-assistance systems (ADAS) and other systems for the purpose of safe driving have been implemented in vehicles. When updating an application to improve its functionality or fix bugs, the quality of the application has traditionally been checked by creating a SIM environment and conducting test drives using test vehicles. When creating a SIM environment to check the quality of an application, the conditions in the real world and the SIM environment do not necessarily match, and if the conditions do not match, this poses the problem of requiring time and cost to make the conditions match. There is also the problem of the time and cost required to collect the data required for the SIM environment in the first place. When checking the quality of an application by conducting test drives using test vehicles, there is the problem of the time and cost required to prepare the test vehicle and conduct the test drives.

[0004] For example, Patent Document 1 discloses a configuration in which a server acquires difference information indicating the difference between feature information and the feature corresponding to the feature information from multiple on-board devices equipped with sensors that measure features, requests raw data, which is measurement data of the feature, from the on-board devices based on the reliability calculated from the multiple difference information, acquires raw data of features that may have changed from the on-board devices, analyzes it in detail, and identifies changes in the feature. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent Publication No. 2021-73547 Summary of the Invention

[0006] The configuration of Patent Document 1 identifies changes in features, but is unable to check the quality of an app, and does not solve the problem of the time and cost required to check the quality of an app, as described above.

[0007] The present disclosure aims to reduce the time and cost required to check the quality of an app, while being able to properly check the quality of the app, and to properly provide apps with guaranteed quality.

[0008] According to one aspect of the present disclosure, the collating unit collates an operation signal of an application distributed from the application distribution device to the in-vehicle device. No. The normal operation determination unit determines whether the app is operating normally based on the result of the comparison by the comparison unit. The operation permission determination unit determines whether the app is operable based on the result of the determination by the normal operation determination unit. If the operation control unit determines that the app is operable, it permits operation of the app.

[0009] The operation signal of the application distributed from the application distribution device to the in-vehicle device No. The system compares the app's operating status and determines whether the app is operating normally based on the results of the comparison. The system determines whether the app can be operated based on the results of the determination of whether the app is operating normally, and if it is determined that the app can be operated, the app is permitted to be operated. The system can determine whether the app is operating normally and determine whether the app can be operated without having to create a SIM environment or conduct test runs using a test vehicle. This reduces the time and cost required to check the quality of an app, while also allowing the quality of the app to be properly checked and quality-guaranteed apps to be properly provided. [Brief explanation of the drawings]

[0010] The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which: [Figure 1] FIG. 1 is a functional block diagram illustrating the overall configuration of an application quality confirmation system according to a first embodiment. [Figure 2] FIG. 2 is a diagram illustrating a shadow mode phase and an operational mode phase; [Figure 3] FIG. 3 is a flowchart illustrating an application registration determination process. [Figure 4] FIG. 4 is a flowchart showing a normal operation determination process. [Figure 5] FIG. 5 is a flowchart showing an operation possibility determination process. [Figure 6] FIG. 6 is a flowchart showing a shadow mode release determination process. [Figure 7] FIG. 7 is a flowchart illustrating a normal operation determination process according to the second embodiment. [Figure 8] FIG. 8 is a flowchart showing an operation possibility determination process. [Figure 9] FIG. 9 illustrates a third embodiment and is a diagram illustrating a shadow mode phase and an operational mode phase. [Figure 10] FIG. 10 shows the fourth embodiment and is a diagram illustrating a shadow mode phase, an operation mode phase, and an operation mode suppression phase. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, several embodiments will be described with reference to the drawings. In the following embodiments, the description of the overlapping parts with the preceding embodiments will be omitted. (First embodiment)

[0012] A first embodiment will be described below with reference to Figs. 1 to 6. As shown in Fig. 1, an application quality confirmation system 1 is configured so that an onboard device 2 mounted on a vehicle and a server 3 (corresponding to an application distribution device) located on the network side can communicate data with each other via a communication network 4 including, for example, the Internet. The vehicle on which the onboard device 2 is mounted may be a vehicle with an automatic driving function or a vehicle without an automatic driving function. A vehicle with an automatic driving function travels by successively switching between automatic driving and manual driving. The onboard device 2 and the server 3 have a multiple-to-one relationship, and the server 3 can communicate data with an unspecified number of onboard devices 2.

[0013] The vehicle-mounted device 2 inputs surrounding information about the vehicle's surroundings, driving information about vehicle driving, and location information about the vehicle's position from various sensors and various electronic control units (ECUs) mounted on the vehicle. The vehicle-mounted device 2 inputs, as surrounding information, camera images of the vehicle's traveling direction captured by an onboard camera, sensor information about the vehicle's surroundings detected by a sensor such as a millimeter-wave sensor, radar information about the vehicle's surroundings detected by a radar, and lidar information about the vehicle's surroundings detected by a lidar (Light Detection and Ranging, Laser Imaging Detection and Ranging). The onboard camera, sensor, radar, and lidar are autonomous sensors, and the surrounding information is information acquired by the autonomous sensors. The camera images include traffic lights, signs, billboards, markings painted on the road surface, stop lines at intersections, pedestrian crossings, diamond-shaped marks at intersections, and the like. The vehicle-mounted device 2 may input, as surrounding information, at least one of the camera images, sensor information, radar information, and lidar information.

[0014] The vehicle-mounted device 2 inputs vehicle speed information detected by a vehicle sensor as driving information. The vehicle-mounted device 2 inputs GNSS information measured by a GNSS (Global Navigation Satellite System) receiver as position information via a gateway 5 serving as a relay device. GNSS is a general term for global positioning and navigation satellite systems, and various systems have been implemented, such as GPS (Global Positioning System), GLONASS (Global Navigation Satellite System), Galileo, BeiDou, and IRNSS (Indian Regional Navigational Satellite System).

[0015] The vehicle-mounted device 2 includes a control unit 6, a data communication unit 7, a probe data storage unit 8, a map data storage unit 9, and an application storage unit 10. The control unit 6 is configured by a microcomputer having a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), and I / O (Input / Output). The microcomputer executes computer programs stored in a non-transitory tangible storage medium to perform processing corresponding to the computer programs and control the overall operation of the vehicle-mounted device 2. The microcomputer has the same meaning as a processor. In the vehicle-mounted device 2, the non-transitory tangible storage medium may share hardware with other computer resources. The probe data storage unit 8, the map data storage unit 9, and the application storage unit 10 may each be configured mainly using one or more non-transitory tangible storage media independently provided for the corresponding data. The probe data storage unit 8, the map data storage unit 9, and the application storage unit 10 may each be configured mainly using a common non-transitory tangible storage medium. The probe data storage unit 8, the map data storage unit 9, and the application storage unit 10 may correspond to one storage medium, or may correspond to a partial storage area of ​​one or more storage media. A storage device may be configured to include at least one of the probe data storage unit 8, the map data storage unit 9, and the application storage unit 10. The storage device may include a circuit for reading and rewriting data.

[0016] The server 3 includes a control unit 11, a data communication unit 12, a probe data storage unit 13, a map data storage unit 14, and an application storage unit 15. The control unit 11 is configured by a microcomputer having a CPU, ROM, RAM, and I / O. The microcomputer executes a computer program stored in a non-transitory tangible storage medium to perform processing corresponding to the computer program and control the overall operation of the server 3. In the server 3, the non-transitory tangible storage medium may also share hardware with other computer resources. The probe data storage unit 13, the map data storage unit 14, and the application storage unit 15 may each be configured mainly using a non-transitory tangible storage medium independently provided for the corresponding data. The probe data storage unit 13, the map data storage unit 14, and the application storage unit 15 may each be configured mainly using a common non-transitory tangible storage medium. The probe data storage unit 13, the map data storage unit 14, and the application storage unit 15 may correspond to a single storage medium or may correspond to a partial storage area of ​​one or more storage media. The storage device may be configured to include at least one of the probe data storage unit 13, the map data storage unit 14, and the application storage unit 15. The storage device may include a circuit for reading and rewriting data.

[0017] In the onboard device 2, when the control unit 6 receives surrounding information, driving information, and position information, it generates probe data from the various input information and stores the generated probe data in the probe data storage unit 8. The probe data is data that includes surrounding information, driving information, and position information, and includes data that indicates the positions, colors, characteristics, relative positional relationships, etc. of traffic lights, signs, billboards, dividing lines, stop lines at intersections, pedestrian crossings, diamond-shaped marks in intersections, etc. that are installed on the road. The probe data also includes data that indicates the road shape, road characteristics, road width, etc., of the road on which the vehicle is traveling.

[0018] The control unit 6 reads out the probe data stored in the probe data storage unit 8 and causes the data communication unit 7 to transmit the read probe data to the server 3, for example, when a predetermined time has elapsed or when the vehicle has traveled a predetermined distance. Instead of using the time or the vehicle's travel distance as the trigger, the control unit 6 may use the reception by the data communication unit 7 of a probe data transmission request transmitted from the server 3 as the trigger, as long as the server 3 is configured to transmit a probe data transmission request to the in-vehicle device 2 at predetermined intervals. Furthermore, the control unit 6 may, for example, when the ignition is turned on, cause the data communication unit 7 to transmit the probe data accumulated in the trip from the previous ignition on to the ignition off to the server 3, or when the ignition is turned off, cause the data communication unit 7 to transmit the probe data accumulated in the trip from the current ignition on to the ignition off to the server 3. When the control unit 6 causes the data communication unit 7 to transmit the probe data to the server 3, the control unit 6 may cause the data communication unit 7 to transmit the probe data to the server 3 in segment units, which are predetermined area units for managing maps, or may cause the data communication unit 7 to transmit the probe data to the server 3 in specified area units that are unrelated to segment units.

[0019] The map data storage unit 9 stores high-precision map data. The map data stored in the map data storage unit 9 includes three-dimensional map information, feature information, road attribute value information, etc. The three-dimensional map information is information including point clouds of road shape and structural feature points. The feature information is information about the shapes and positions of traffic lights, signs, billboards, dividing lines, stop lines at intersections, pedestrian crossings, diamond-shaped marks within intersections, etc. The road attribute value information is information about road lanes, such as the number of lanes and whether or not there are dedicated right-turn lanes. The map data stored in the map data storage unit 9 is updated successively by downloading map data stored in a map data storage unit 14 of the server 3 (described later) from the server 3 to the in-vehicle device 2.

[0020] The application storage unit 10 stores various applications that contribute to, for example, ADAS. The applications stored in the application storage unit 10 include, for example, adaptive cruise control (ACC), lane change assist (LCA), lane departure warning (LDW), lane keeping assist (LKA), forward collision warning (FCW), traffic sign recognition (TSR), and advanced emergency braking system (AEBS). The applications stored in the application storage unit 10 are updated successively by downloading applications stored in an application storage unit 15 of a server 3 (described later) from the server 3 to the in-vehicle device 2.

[0021] In the server 3, the map data storage unit 14 stores high-precision map data. The map data stored in the map data storage unit 14 has a larger capacity than the map data stored in the map data storage unit 9 of the vehicle-mounted device 2, and reflects information on a wider area. The control unit 11 receives probe data transmitted from the vehicle-mounted device 2 via the data communication unit 12 and stores the received probe data in the probe data storage unit 13. The control unit 11 reads out the probe data stored in the probe data storage unit 13 and sequentially updates the map data stored in the map data storage unit 14 with the read probe data. In other words, the map data stored in the map data storage unit 14 is integrated map data that is generated by sequentially reflecting multiple pieces of probe data.

[0022] The application storage unit 15 stores various applications that contribute to, for example, ADAS, etc. The applications stored in the application storage unit 15 are updated successively as the application functions are improved, bugs are fixed, and so on.

[0023] In the above-described configuration, the quality of an application downloaded from the server 3 to the in-vehicle device 2 and operated by the in-vehicle device 2 must be guaranteed. The in-vehicle device 2 and server 3 have the following functions to verify the quality of the application. As shown in FIG. 2, the in-vehicle device 2 installed in a vehicle traveling on public roads executes an application in two phases: a shadow mode phase and an operational mode phase. The shadow mode phase is a phase in which the application's operational signals are not reflected in real-world vehicle control when the application is executed. For example, in the case of an application that controls ACC, the application that controls ACC is executed on software, but the ACC operational signals output from the application are not transmitted to the control mechanisms involved in ACC control, thereby preventing ACC control. On the other hand, the operational mode phase is a phase in which the application's operational signals are reflected in real-world vehicle control when the application is executed. For example, in the case of an application that controls ACC, the application that controls ACC is executed on software, and the ACC operational signals output from the application are transmitted to the control mechanisms involved in ACC control, thereby controlling ACC.

[0024] The server 3 manages whether multiple apps require pre-operational checks and registers shadow mode for apps that require pre-operational checks. Apps that require pre-operational checks are apps for which there is no guarantee that their operation signals will be reflected in real-world vehicle control. The server 3 manages apps that have registered shadow mode as apps that are subject to shadow mode and transmits the apps that are subject to shadow mode to the unspecified number of in-vehicle devices 2. When the unspecified number of in-vehicle devices 2 receive apps that are subject to shadow mode transmitted from the server 3 and execute the received apps that are subject to shadow mode, they compare the app's operation signals with the driver's actions and the detection results of the autonomous sensors installed in the vehicle and determine whether the apps are operating normally based on the comparison results. When the unspecified number of in-vehicle devices 2 determine that the apps are operating normally, they transmit a determination result indicating that the apps are operating normally to the server 3. When they determine that the apps are not operating normally, they transmit a determination result indicating that the apps are not operating normally to the server 3.

[0025] When the server 3 receives the determination results of whether the app is operating normally or not transmitted from the unspecified number of in-vehicle devices 2, the server 3 statistically analyzes the received determination results to determine whether the app is operable. If the server 3 determines that the app is operable through the statistical analysis, the server 3 transitions from the shadow mode phase to the operation mode phase, transmits a release signal to the unspecified number of in-vehicle devices 2, releases the shadow mode of the app, and starts operation of the app on the in-vehicle devices 2. For an app whose shadow mode has been registered in this way, the server 3 determines whether the app is operating normally in the shadow mode phase by the unspecified number of in-vehicle devices 2, statistically analyzes the determination results of whether the app is operating normally by the unspecified number of in-vehicle devices 2, and if the server 3 determines that the app is operable through the statistical analysis, the server 3 releases the shadow mode and the app is operated by the unspecified number of in-vehicle devices 2 in the operation mode phase.

[0026] The in-vehicle device 2 and server 3 have the following functions as a configuration for realizing the above-mentioned processing. In the in-vehicle device 2, the control unit 6 has a matching unit 6a, a normal operation determination unit 6b, a determination result transmission unit 6c, and a vehicle control unit 6d. The matching unit 6a matches the operation signal of the app with the driver's actions and the detection results of the autonomous sensors installed in the vehicle. The normal operation determination unit 6b determines whether the app is operating normally based on the matching result of the matching unit 6a.

[0027] Specifically, for an app that controls ACC, the collating unit 6a collates the acceleration / deceleration and vehicle speed values, the timing of acceleration / deceleration, etc. calculated by the app with the presence or absence of the driver's brake or accelerator operation, the timing of those operations, and the actual acceleration / deceleration and vehicle speed values ​​resulting from those operations. The normal operation determining unit 6b determines that the app is operating normally and determines whether the app is operating normally if the driver has operated the brake or accelerator and the timing and numerical differences are within a specified range. In other words, if there is a relatively small discrepancy between the driver's intended acceleration / deceleration, vehicle speed, and operation timing and the operation of the app that controls ACC, the normal operation determining unit 6b determines that the app is not operating normally and determines whether the app is operating normally. On the other hand, if the driver has not operated the brake or accelerator, or if the driver has operated the brake or accelerator but the timing and numerical differences are not within a specified range, the normal operation determining unit 6b determines that the app is not operating normally and determines whether the app is operating normally. That is, if there is a relatively large discrepancy between the driver's intended acceleration / deceleration, vehicle speed, and operation timing and the operation of the app that controls the ACC, the normal operation determination unit 6b determines whether the app is operating normally. However, in places where the speed limit has changed or where there is acceleration / deceleration due to a change in the set vehicle speed by the driver, the normal operation determination unit 6b does not determine whether the app is operating normally.

[0028] For apps that control LCA, LDW, and LKA, the collating unit 6a collates the steering angle values, steering timing, etc. calculated by the app with the presence or absence of a steering operation by the driver, the timing of the steering, and the actual steering angle values ​​resulting from the operation. If the driver performs a steering operation and the timing and numerical difference are within a specified range, the normal operation determining unit 6b determines that the app is operating normally and determines whether the app is operating normally. On the other hand, if the driver does not perform a steering operation, or if the driver performs a steering operation but the timing and numerical difference are not within a specified range, the normal operation determining unit 6b determines that the app is not operating normally and determines whether the app is operating normally. For LCA apps, if the driver does not perform a steering operation after the turn signal is activated, the normal operation determining unit 6b determines that the app is operating normally and determines whether the app is operating normally.

[0029] For an app that controls FCW, the collating unit 6a collates the warning display and warning sound output by the app with the presence or absence of the driver's brake operation, the timing of the operation, etc. The normal operation determining unit 6b determines that the app is operating normally if the driver has operated the brakes and the difference in timing is within a specified range, and determines whether the app is operating normally. On the other hand, if the driver has not operated the brakes, or if the driver has operated the brakes but the difference in timing is not within a specified range, the normal operation determining unit 6b determines that the app is not operating normally and determines whether the app is operating normally.

[0030] For an app that controls the TSR, the collating unit 6a collates the contents of the sign detected by the app with the driver's behavior and vehicle state. If the contents of the sign detected by the app indicate a speed limit, the normal operation determining unit 6b determines that the app is operating normally if the driver brakes or accelerates and the difference between the speed limit indicated by the sign and the actual vehicle speed is within a specified range, and determines whether the app is operating normally. On the other hand, if the driver does not brake or accelerate, or if the driver brakes or accelerates but the difference is not within a specified range, the normal operation determining unit 6b determines that the app is not operating normally and determines whether the app is operating normally. If the contents of the sign detected by the app indicate a stop sign, the normal operation determining unit 6b determines that the app is operating normally if the driver brakes and the vehicle decelerates and stops, and determines whether the app is operating normally. On the other hand, if the driver does not brake or brakes and the vehicle decelerates but does not stop, the normal operation determining unit 6b determines that the app is not operating normally and determines whether the app is operating normally. When the content of the sign detected by the app indicates no entry, and the vehicle is not traveling in a no entry direction, the normal operation determination unit 6b determines that the app is operating normally and determines whether the app is operating normally. On the other hand, if the vehicle is traveling in a no entry direction, the normal operation determination unit 6b determines that the app is not operating normally and determines whether the app is operating normally.

[0031] For an application that controls the AEBS, the collating unit 6a collates the collision mitigation operation signal calculated by the application with whether or not the driver has applied the brakes. If the driver has applied the brakes, the normal operation determining unit 6b determines that the application is operating normally and determines whether the application is operating normally. On the other hand, if the driver has not applied the brakes, the normal operation determining unit 6b determines that the application is not operating normally and determines whether or not the application is operating normally.

[0032] The determination result transmission unit 6c causes the data communication unit 7 to transmit the determination result of the normal operation determination unit 6, whether the app is operating normally or not, to the server 3. In this case, the determination result transmission unit 6c causes the data communication unit 7 to transmit, in addition to the determination result of the normal operation determination unit 6, whether the app is operating normally or not, information about the object to be determined, information about the calculation result calculated in the determination process, information about vehicle control, information about the surrounding environment, and the like, to the server 3. Furthermore, the determination result transmission unit 6c may select the determination result of normal operation or not to be transmitted from the data communication unit 7 to the server 3 based on the image recognition result. The determination result transmission unit 6c may determine the accuracy of the image recognition by determining the recognition level by image recognition as a threshold, and select whether to transmit the determination result of normal operation or not from the data communication unit 7 to the server 3. The determination result transmission unit 6c may not transmit from the data communication unit 7 to the server 3 the determination result of whether or not the vehicle is operating normally for features whose recognition level is below a threshold, but may transmit from the data communication unit 7 to the server 3 the determination result of whether or not the vehicle is operating normally for features whose recognition level is equal to or higher than the threshold. Furthermore, the determination result of whether or not the vehicle is operating normally to be transmitted from the data communication unit 7 to the server 3 may be selected based on the detection result of an autonomous sensor, the operating state of on-board equipment, etc., in addition to the image recognition result. For example, since the detection result of the autonomous sensor may be degraded during rainfall or snowfall, the determination result transmission unit 6c may not transmit from the data communication unit 7 to the server 3 the determination result of whether or not the vehicle is operating normally when the windshield wipers are operating or the rain sensor is detecting rain, but may transmit the determination result of whether or not the vehicle is operating normally when the windshield wipers are not operating or the rain sensor is not detecting rain.

[0033] The determination result transmission unit 6c causes the data communication unit 7 to transmit to the server 3, as information related to the determination target, the version of the map used, the mesh (plot) number, the lane line ID of the lane line being determined, the distance from the lane line endpoint, and other indicators that can identify which part of the lane line has been determined. The determination result transmission unit 6c causes the data communication unit 7 to transmit to the server 3, as information related to the calculation results calculated in the determination process, the coordinates of the map data to be inspected in the vehicle position coordinate system, the coordinates of the recognition points used in the determination, the calculated distance, the x-component of that distance, the average and variance of that x-component, the slope of the data point sequence, the number of map data points and recognition points, and the like. The determination result transmission unit 6c causes the data communication unit 7 to transmit to the server 3, as information related to vehicle control, the absolute coordinates of the vehicle position, the steering angle, the accelerator state, the brake state, the detection results of the autonomous system sensors, and the like. The determination result transmission unit 6c causes the data communication unit 7 to transmit to the server 3, as information related to the surrounding environment, the presence or absence of a preceding vehicle, the presence or absence of a following vehicle, and the rainfall or snowfall conditions, and the like.

[0034] The vehicle control unit 6d controls the vehicle in accordance with the driver's actions and the detection results of the autonomous sensors mounted on the vehicle.

[0035] In the server 3, the control unit 11 has a determination result receiving unit 11a, an operation feasibility determining unit 11b, and an operation control unit 11c. Each of these units 11a to 11c corresponds to a part of the function executed by the application quality confirmation program. That is, the control unit 11 performs the functions of each of the units 11a to 11c by executing a part of the application quality confirmation program.

[0036] The determination result receiving unit 11a receives the determination result of whether the application operates normally or not transmitted from the in-vehicle device 2. The operation feasibility determining unit 11b determines whether the application is operable or not based on the determination result of whether the application operates normally or not transmitted from the in-vehicle device 2. The operation feasibility determining unit 11b performs a screening process and a statistical process by majority vote as a method for determining whether the application is operable or not.

[0037] As a screening process, the operation feasibility determination unit 11b uses information on the calculation results calculated in the determination process, information on vehicle control, and information on the surrounding environment to exclude determination results that are expected to have a low accuracy rate from the determination results received from the on-board device 2. Specifically, the operation feasibility determination unit 11b excludes determination results if, for example, the determination results include information that autonomous driving control is not being performed appropriately, information on the status information of each sensor includes information on a malfunction, or information on the presence of a preceding vehicle in a close distance ahead. As a statistical process by majority vote, when there are multiple determination results after the above-mentioned screening process, the operation feasibility determination unit 11b calculates which of the determination results indicating normal operation or normal operation is in the majority, and adopts the determination result that is determined to be the majority.

[0038] When the operation possibility determination unit 11b determines that the application subject to shadow mode can be operated, the operation control unit 11c transmits a cancellation signal to the vehicle-mounted device 2 to cancel the shadow mode and start operation of the application on the vehicle-mounted device 2. On the other hand, when the operation possibility determination unit 11b determines that the application subject to shadow mode cannot be operated, the operation control unit 11c does not transmit a cancellation signal to the vehicle-mounted device 2, does not cancel the shadow mode, and does not start operation of the application on the vehicle-mounted device 2.

[0039] Next, the operation of the above-described configuration will be described with reference to Fig. 3 to Fig. 6. The application registration determination process performed by the server 3, the normal operation determination process performed by the in-vehicle device 2, the operation determination process performed by the server 3, and the shadow mode release determination process performed by the in-vehicle device 2 will be described in order.

[0040] (1-1) Application registration determination process performed by server 3 (see Figure 3) In the server 3, when the control unit 11 starts the application registration determination process, it determines whether or not an application to be updated is registered (A1). If the control unit 11 determines that an application to be updated is not registered (A1: NO), it ends the application registration determination process. If the control unit 11 determines that an application to be updated is registered because the application to be updated has been stored in the application storage unit 15 due to, for example, an application function improvement or bug repair (A1: YES), it determines whether or not the application to be updated is an application to be used in shadow mode (A2). That is, the control unit 11 determines whether or not the application to be updated is an application that requires operation confirmation before operation.

[0041] If the control unit 11 determines that the application to be updated is a shadow mode application (A2: YES), it distributes the application to be updated to the in-vehicle device 2 as an application that is subject to shadow mode (A3), and ends the application registration determination process. On the other hand, if the control unit 11 determines that the application to be updated is not a shadow mode application (A2: NO), it distributes the application to be updated to the in-vehicle device 2 as an application that is not subject to shadow mode (A4), and ends the application registration determination process. The application distributed from the server 3 to the in-vehicle device 2 is stored in the application storage unit 10.

[0042] (1-2) Normal Operation Judgment Process Performed by the In-Vehicle Device 2 (See Figure 4) In the in-vehicle device 2, when the control unit 6 starts the normal operation determination process, it refers to the application storage unit 10 and determines whether there is an application that is subject to shadow mode (B1). If the control unit 6 determines that there is no application that is subject to shadow mode (B1: NO), it ends the normal operation determination process. If the control unit 6 determines that an application that is subject to shadow mode is stored in the application storage unit 10 and that there is an application that is subject to shadow mode (B1: YES), it compares the vehicle position with map data and determines whether the vehicle position is within the determination target area (B2). If the control unit 6 determines that the vehicle position is not within the determination target area (B2: NO), it ends the normal operation determination process.

[0043] When the control unit 6 determines that the vehicle position is within the determination target area (B2: YES), it determines whether the execution conditions for the shadow mode target app are met (B3). That is, the control unit 6 determines whether the execution conditions for the shadow mode target app are met based on the vehicle position, acceleration / deceleration, vehicle speed, the distance from the vehicle to the preceding vehicle, the surrounding environment of the vehicle, etc. When the control unit 6 determines that the execution conditions for the shadow mode target app are not met (B3: NO), it ends the normal operation possibility determination process.

[0044] When the control unit 6 determines that the execution conditions for the shadow mode app are met (B3: YES), it executes the shadow mode app and acquires an operation signal of the app when executing the shadow mode app (B4). The control unit 6 acquires the driver's actions (B5) and acquires the detection results of the autonomous system sensors (B6). The control unit 6 compares the operation signal of the app with the driver's actions and the detection results of the autonomous system sensors, and determines whether the app is operating normally (B7). That is, as described above, for an app that performs ACC control, for example, the control unit 6 compares the acceleration / deceleration and vehicle speed values, acceleration / deceleration timing, etc. calculated by the app with the presence or absence of the driver's brake or accelerator operation, the timing of those operations, and the actual acceleration / deceleration and vehicle speed values ​​resulting from those operations. If the discrepancy is relatively small, the control unit 6 determines whether the app is operating normally. If the discrepancy is relatively large, the control unit 6 determines whether the app is operating normally. The same applies to the application that controls LCA, LDW, and LKA, the application that controls FCW, the application that controls TSR, and the application that controls AEBS.

[0045] If the control unit 6 determines that the discrepancy between the operation signal of the app and the detection results of the driver's action and the autonomous sensor is relatively small and that the app is operating normally (B7: YES), it causes the data communication unit 7 to transmit a determination result indicating that the app is operating normally to the server 3 (B8) and terminates the normal operation determination process. On the other hand, if the control unit 6 determines that the discrepancy between the driver's action and the detection results of the autonomous sensor is relatively large and that the app is not operating normally (B7: NO), it causes the data communication unit 7 to transmit a determination result indicating that the app is not operating normally to the server 3 (B9) and terminates the normal operation determination process. If the execution conditions for multiple shadow mode target apps are met, the control unit 6 performs the above processes in parallel to determine whether each of the multiple apps is operating normally.

[0046] (1-3) Operation feasibility determination process performed by server 3 (see Figure 5) In the server 3, when the control unit 11 starts the operation feasibility determination process, it determines whether or not it has received a determination result indicating whether or not the application is operating normally from the in-vehicle device 2 (A11, corresponding to the determination result receiving step). When the control unit 11 determines that it has not received a determination result indicating whether or not the application is operating normally from the in-vehicle device 2 (A11: NO), it ends the operation feasibility determination process. When the control unit 11 determines that it has received a determination result indicating whether or not the application is operating normally from the in-vehicle device 2 (A11: YES), it determines whether or not the determination result is for the application to be determined (A12). When the control unit 11 determines that the determination result is not for the application to be determined (A12: NO), it ends the operation feasibility determination process.

[0047] When the control unit 11 determines that the result is for the application to be determined (A12: YES), it performs a screening process (A13), performs statistical processing (A14), and determines whether the application is operable (A15, which corresponds to the operation feasibility determination procedure). The control unit 11 determines whether the application is operable by, for example, monitoring the ratio of determination results indicating that the application can operate normally or the number of determination results indicating that the application can operate normally. If the ratio of determination results indicating that the application can operate normally does not reach a predetermined value and the number of determination results indicating that the application can operate normally does not reach a predetermined value, the control unit 11 determines that the application is operable (A15: NO) and ends the operation feasibility determination process.

[0048] When the ratio of determination results indicating that the application can operate normally reaches a predetermined value, or when the number of determination results indicating that the application can operate normally reaches a predetermined value, the control unit 11 determines that the application can be operated (A15: YES). When the control unit 11 determines that the application can be operated, it cancels the registration of the shadow mode (A16, corresponding to the operation control procedure), causes the data communication unit 12 to transmit a cancellation signal to the in-vehicle device 2 (A17), and ends the operation possibility determination process.

[0049] (1-4) Shadow mode release determination process performed by the in-vehicle device (see Figure 6) In the vehicle-mounted device 2, when the control unit 6 starts the shadow mode release determination process, it determines whether or not a release signal has been received from the server 3 (B11). If the control unit 6 determines that a release signal has not been received from the server 3 (B11: NO), it ends the shadow mode release determination process. If the control unit 6 determines that a release signal has been received from the server 3 (B11: YES), it releases shadow mode (B12) and ends the shadow mode release determination process.

[0050] As described above, the first embodiment can achieve the following advantageous effects. When the in-vehicle device 2 receives a shadow mode-enabled app transmitted from the server 3, it compares the app's operation signal with the driver's actions and the detection results of the autonomous sensors installed in the vehicle, determines whether the app is operating normally based on the comparison results, and transmits the determination result indicating whether the app is operating normally to the server 3. When the server 3 receives the determination result indicating whether the app is operating normally transmitted from the in-vehicle device 2, it determines whether the app is operational based on the received determination result, and if it determines that the app is operational, it permits operation of the app. Whether the app is operating normally and whether the app is operational can be determined without creating a SIM environment or conducting test drives using a test vehicle. This reduces the time and cost required to check the quality of the app, while properly checking the quality and enabling the appropriate provision of quality-guaranteed apps.

[0051] The system uses apps that are currently registered in shadow mode as the target of comparison, and if it determines that the app can be used, it cancels shadow mode and starts using the app. If it determines that the app cannot be used, it does not start using the app. It is possible to determine whether to start using the app by using the app before it is used as the target of comparison.

[0052] (Second embodiment) Next, a second embodiment will be described with reference to Figures 7 and 8. In the first embodiment, the vehicle-mounted device 2 stores map data, but in the second embodiment, the vehicle-mounted device 2 does not store map data.

[0053] (2-1) Normal Operation Judgment Process Performed by the In-Vehicle Device 2 (See FIG. 7) In the vehicle-mounted device 2, when the control unit 6 determines that there is an application that is subject to shadow mode (B1: YES), it determines whether or not the execution conditions for the application that is subject to shadow mode are met (B3) without performing step B2 described in the first embodiment. When the control unit 6 determines that the execution conditions for the application that is subject to shadow mode are met (B3: YES), it performs steps B4 to B9 described in the first embodiment.

[0054] (2-2) Operational feasibility determination process performed by server 3 (see Figure 8) In the server 3, the control unit 11 determines that it has received a judgment result indicating whether the app is operating normally from the in-vehicle device 2 (A11: YES), and if it determines that the judgment result is for the app to be judged (A12: YES), it identifies a position on the map data based on the vehicle position received from the in-vehicle device 2 (A21). That is, the control unit 11 identifies the position on the map data where the in-vehicle device 2 executed the app that is subject to shadow mode, and performs steps A13 to A17 described in the first embodiment.

[0055] As described above, according to the second embodiment, the following advantageous effects can be obtained: Even if the vehicle-mounted device 2 does not store map data, the server 3 can identify the position on the map data where the vehicle-mounted device 2 executed an app that is subject to shadow mode, thereby obtaining advantageous effects similar to those of the first embodiment.

[0056] (Third embodiment) Next, a third embodiment will be described with reference to Fig. 9. In the first and second embodiments, the in-vehicle device 2 receives a shadow mode target app transmitted from the server 3 during the process of determining whether the app is operating normally, and if it determines that the app is operating normally, transmits a determination result indicating that the app is operating normally to the server 3. If it determines that the app is not operating normally, it transmits a determination result indicating that the app is not operating normally to the server 3. However, in the third embodiment, only the determination result indicating that the app is not operating normally is transmitted to the server 3.

[0057] 9, when the unspecified number of in-vehicle devices 2 receive an app that is subject to shadow mode transmitted from the server 3, when executing the received app that is subject to shadow mode, the unspecified number of in-vehicle devices 2 compares the operation signal of the app with the driver's actions and the detection results of the autonomous sensors mounted on the vehicle, and determines whether the app is operating normally based on the comparison results, as in the first embodiment. If the unspecified number of in-vehicle devices 2 determine that the app is operating normally, they do not transmit a determination result indicating that the app is operating normally to the server 3, but if they determine that the app is not operating normally, they transmit a determination result indicating that the app is not operating normally to the server 3.

[0058] When the server 3 receives the determination results of whether the app is operating normally transmitted from an unspecified number of in-vehicle devices 2, the server 3 accumulates the received determination results of whether the app is operating normally and determines whether the frequency of the determination results of whether the app is operating normally is equal to or greater than a threshold. When the server 3 determines that the frequency of the determination results of whether the app is operating normally is equal to or greater than the threshold, the server 3 transmits a request signal to the unspecified number of in-vehicle devices 2.

[0059] The unspecified number of in-vehicle devices 2 receive the request signal transmitted from the server 3, and when they determine that the app can operate normally, they transmit a determination result indicating that the app can operate normally to the server 3. When they determine that the app cannot operate normally, they transmit a determination result indicating that the app cannot operate normally to the server 3. After this, as in the first embodiment, when the server 3 receives the determination results indicating that the app can operate normally or cannot operate normally transmitted from the unspecified number of in-vehicle devices 2, it statistically analyzes the received determination results and determines whether the app can be operated.

[0060] As described above, according to the third embodiment, the following advantageous effects can be obtained. When the frequency of the determination results of whether an app is operating normally transmitted from the in-vehicle device 2 is equal to or greater than a threshold, the server 3 transmits a request signal to the in-vehicle device 2. When the in-vehicle device 2 receives the request signal transmitted from the server 3, it transmits a determination result indicating whether the app is operating normally to the server 3. By transmitting only the determination result of whether the app is operating normally from the in-vehicle device 2 to the server 3 and, once the frequency of the determination results of whether the app is operating normally or not is equal to or greater than the threshold, transmitting both the determination result indicating whether the app is operating normally and the determination result indicating whether the app is operating normally from the in-vehicle device 2 to the server 3, it is possible to suppress an increase in the amount of data communication from the in-vehicle device 2 to the server 3 and perform a detailed investigation of the app determined to be operating normally.

[0061] (Fourth embodiment) Next, a fourth embodiment will be described with reference to Fig. 10. In the first to third embodiments, the vehicle-mounted device 2 determines whether an application operates normally only in the shadow mode phase, and does not determine whether an application operates normally in the operation mode phase. However, in the fourth embodiment, the vehicle-mounted device 2 determines whether an application operates normally also in the operation mode phase.

[0062] 10, even after the unspecified number of in-vehicle devices 2 has started operation after canceling shadow mode by receiving a cancel signal transmitted from the server 3, the unspecified number of in-vehicle devices 2 compares the operation signal of the app with the driver's actions and the detection results of the autonomous sensors mounted on the vehicle, and determines whether the app is operating normally based on the comparison results. When the unspecified number of in-vehicle devices 2 determines that the app is operating normally, it transmits a determination result indicating that the app is operating normally to the server 3, and when it determines that the app is not operating normally, it transmits a determination result indicating that the app is not operating normally to the server 3.

[0063] When the server 3 receives the determination results of whether the app is operating normally or not transmitted from the unspecified number of in-vehicle devices 2, the server 3 statistically analyzes the received determination results to determine whether the app is operational. If the server 3 determines that the app is operational through the statistical analysis, it continues operation of the app. If the server 3 determines that the app is not operational through the statistical analysis, it transmits a continuation suppression signal to the unspecified number of in-vehicle devices 2 to suppress continuation of operation of the app. In this way, the app that has been released from shadow mode and started to operate is determined to be operating normally by the unspecified number of in-vehicle devices 2 in the operation mode phase, and the server 3 statistically analyzes the determination results of whether the app is operating normally by the unspecified number of in-vehicle devices 2. If the statistical analysis determines that the app is not operational, continuation of operation by the unspecified number of in-vehicle devices 2 is suppressed.

[0064] As described above, according to the fourth embodiment, the following advantageous effects can be obtained. An application after operation has started is used as a comparison target, and if it is determined that the application can be operated after operation has started, the operation of the application is continued, whereas if it is determined that the application cannot be operated after operation has started, the continuation of the application is restricted. It is possible to determine whether to continue the operation of an application by using the application after operation as a determination target.

[0065] (Other embodiments) Although the present disclosure has been described with reference to the embodiments, it is understood that the present disclosure is not limited to the embodiments or structures. The present disclosure also encompasses various modifications and modifications within the scope of equivalents. In addition, various combinations and forms, as well as other combinations and forms including only one element, more than one element, or less than one element, are also within the scope and spirit of the present disclosure.

[0066] The control unit and the method described herein may be implemented by a special-purpose computer configured by configuring a processor and memory programmed to perform one or more functions embodied in a computer program. Alternatively, the control unit and the method described herein may be implemented by a special-purpose computer configured by configuring a processor with one or more dedicated hardware logic circuits. Alternatively, the control unit and the method described herein may be implemented by one or more special-purpose computers configured by combining a processor and memory programmed to perform one or more functions with a processor configured with one or more hardware logic circuits. Furthermore, the computer program may be stored as instructions executed by a computer on a computer-readable non-transitory tangible storage medium.

[0067] In addition to the inventions set forth in the claims, the present disclosure includes the following inventions. [1] An application quality confirmation system (1) for confirming the quality of an application distributed from an application distribution device to an in-vehicle device, The application distribution device distributes an operation signal of the application to the in-vehicle device. No. a matching unit (6a) for matching; a normal operation determination unit (6b) that determines whether the application is operating normally based on the collation result of the collation unit; an operation possibility determination unit (11b) that determines whether the application is operable based on a determination result of the normal operation determination unit; an operation control unit (11c) that permits operation of the application when it is determined that the application is operable; 、 the verification unit verifies an operation signal of an application registered in shadow mode as a verification target, and When it is determined that the application registered in the shadow mode is operable, the operation control unit cancels the shadow mode and permits the application to start operating. App quality check system.

[0068] [2] The collation unit ,before The operation signal of the application registered in the shadow mode is compared with at least one of the driver's operation and the detection result of the autonomous system sensor mounted on the vehicle; The system for verifying the quality of an app described in [1], wherein the operation control unit, when determining that the app registered in shadow mode is operable, cancels shadow mode and allows the app to start operating.

[0069] [3] The application quality confirmation system according to [2], wherein the operation control unit, when determining whether or not the application registered in the shadow mode is in operation, inhibits the start of operation of the application.

[0070] [4] The verification unit is configured to verify an application after the start of operation, and No. Check and The quality confirmation system for an app described in any one of [1] to [3], wherein the operation control unit allows the app to continue operating when it determines that the app can be operated after the start of operation.

[0071] [5] The application quality confirmation system according to [4], wherein the operation control unit prevents the continuation of the application when it determines whether the application is to be operated after the start of operation.

[0072] [6] the in-vehicle device includes the collating unit, the normal operation determining unit, and a determination result transmitting unit (6c) that transmits a determination result as to whether the application is operating normally to the application distribution device; The application distribution device is an application quality confirmation system described in any one of [1] to [5], which includes a judgment result receiving unit (11a) that receives a judgment result of whether the application is operating normally, the operation feasibility judgment unit, and the operation control unit. [7] The quality confirmation system for an application described in [6], wherein the judgment result transmission unit selects the judgment result of whether the application is operating normally or not to be transmitted to the application distribution device based on at least one of the image recognition result, the detection result of an autonomous system sensor, and the operating status of an in-vehicle device.

[0073] [8] The determination result transmission unit does not transmit a determination result of whether the application can operate normally to the application distribution device until it determines that a predetermined condition is met, but transmits only a determination result of whether the application can operate normally to the application distribution device, and after determining that the predetermined condition is met, transmits both the determination result of whether the application can operate normally and the determination result of whether the application can operate normally to the application distribution device. 6 ] App quality verification system described in.

[0074] [ 9 ] The determination result transmission unit determines that the predetermined condition is met when the application distribution device determines that the frequency of the determination result of whether the application is operating normally is equal to or greater than a threshold value. 8 ] App quality verification system described in.

[0075] [ 10 ] An application distribution device that distributes an application to an in-vehicle device, The operation signal of the application delivered to the in-vehicle device The number is a determination result receiving unit (11a) that receives a determination result as to whether the application is operating normally based on the result of the comparison; an operation possibility determination unit (11b) that determines whether the application is operable based on a determination result of whether the application is operating normally; an operation control unit (11c) that permits operation of the application when it is determined that the application is operable; 、 The operation control unit performs a comparison of an application registered in shadow mode, compares an operation signal of the application registered in shadow mode, and, when it determines that the application registered in shadow mode is operable, cancels shadow mode and permits the application to start operating. App distribution device.

[0076] [ 11 ] A control unit (11) of an application distribution device (3) that distributes an application to an in-vehicle device, The operation signal of the application delivered to the in-vehicle device The number is a determination result receiving step of receiving a determination result of whether the application is operating normally based on the result of the comparison; an operation possibility determination step of determining whether the application is operable based on a determination result of whether the application is operating normally; When it is determined that the application is operable, an operation control procedure for permitting the operation of the application is executed. 、 In the operation control procedure, an application registered in shadow mode is subjected to comparison, an operation signal of the application registered in shadow mode is compared, and if it is determined that the application registered in shadow mode is operable, the shadow mode is cancelled and the operation of the application is permitted. App quality review program.

Claims

1. An application quality confirmation system (1) for confirming the quality of an application distributed from an application distribution device to an in-vehicle device, a verification unit (6a) that verifies an operation signal of the application distributed from the application distribution device to the in-vehicle device; a normal operation determination unit (6b) that determines whether the application is operating normally based on the collation result of the collation unit; an operation possibility determination unit (11b) that determines whether the application is operable based on a determination result of the normal operation determination unit; an operation control unit (11c) that permits operation of the application when it is determined that the application is operable; the verification unit verifies an operation signal of an application registered in shadow mode as a verification target, and The system for verifying the quality of an app wherein, when it is determined that the application registered in shadow mode is operable, the operation control unit cancels shadow mode and permits the application to start operating.

2. the verification unit verifies an operation signal of the application registered in the shadow mode with at least one of an operation of the driver and a detection result of an autonomous sensor mounted on the vehicle; 2. The application quality confirmation system according to claim 1, wherein the operation control unit, when determining that the application registered in the shadow mode is operable, cancels the shadow mode and permits the application to start operating.

3. The application quality confirmation system according to claim 2 , wherein the operation control unit, when determining whether or not the application registered in the shadow mode is to be operated, inhibits the start of operation of the application.

4. the verification unit is configured to verify an application that has been in operation since the application started, and verify an operation signal of the application that has been in operation since the application started, The application quality confirmation system according to claim 1 , wherein the operation control unit allows the application to continue to be operated if it is determined that the application is operable after the start of operation.

5. The application quality confirmation system according to claim 4 , wherein the operation control unit, when determining whether or not the application is to be operated after the start of operation, prevents the continuation of the operation of the application.

6. the in-vehicle device includes the collating unit, the normal operation determining unit, and a determination result transmitting unit (6c) that transmits a determination result of whether the application is operating normally to the application distribution device; 6. The application quality confirmation system according to claim 1, wherein the application distribution device includes a judgment result receiving unit (11a) that receives a judgment result as to whether the application is operating normally, the operation feasibility determination unit, and the operation control unit.

7. 7. The application quality confirmation system according to claim 6, wherein the determination result transmission unit selects the determination result of whether the application is operating normally to be transmitted to the application distribution device based on at least one of an image recognition result, a detection result of an autonomous sensor, and an operating state of an in-vehicle device.

8. 7. The application quality verification system according to claim 6, wherein the determination result transmission unit does not transmit a determination result of whether the application can operate normally to the application distribution device until it determines that a predetermined condition is met, but transmits only a determination result of whether the application can operate normally to the application distribution device, and after determining that the predetermined condition is met, transmits both the determination result of whether the application can operate normally and the determination result of whether the application cannot operate normally to the application distribution device.

9. 9. The application quality confirmation system according to claim 8, wherein the determination result transmission unit determines that the predetermined condition is met when the application distribution device determines that the frequency of the determination result indicating whether the application is operating normally is equal to or greater than a threshold value.

10. An application distribution device that distributes an application to an in-vehicle device, a determination result receiving unit (11a) that receives a determination result of whether the application is operating normally based on a comparison result obtained by comparing an operation signal of the application delivered to the in-vehicle device; an operation possibility determination unit (11b) that determines whether the application is operable or not based on a determination result of whether the application is operating normally; an operation control unit (11c) that permits operation of the application when it is determined that the application is operable; The operation control unit of the app distribution device performs comparison with an app that is registered in shadow mode, compares an operation signal of the app that is registered in shadow mode, and, if it determines that the app that is registered in shadow mode can be operated, cancels shadow mode and allows the app to start operating.

11. A control unit (11) of an application distribution device (3) that distributes an application to an in-vehicle device, a determination result receiving step of receiving a determination result of whether the application is operating normally based on a comparison result obtained by comparing the operation signal of the application delivered to the in-vehicle device; an operation possibility determination step of determining whether the application is operable based on a determination result of whether the application is operating normally; If it is determined that the application is operable, an operation control procedure is executed to permit the operation of the application; In the operation control procedure, an application registered in shadow mode is subject to comparison, an operation signal of the application registered in shadow mode is compared, and if it is determined that the application registered in shadow mode is operable, the application quality confirmation program cancels shadow mode and allows the application to begin operation.

Images