A vehicle OTA upgrade safety control method, system and storage medium based on image scene recognition

Abstract

The present invention discloses a vehicle OTA upgrade safety control method, system and storage medium based on image scene recognition. The method includes: after the upgrade package is downloaded, the vehicle computer calls the camera to obtain the external environment information of the vehicle; the vehicle computer sends the obtained external environment information to the background for image scene recognition, and obtains the result of the background recognition; the vehicle computer obtains the internal environment information of the vehicle; the vehicle computer determines whether the upgrade conditions are met according to the result of the background recognition and the internal environment information; if the upgrade conditions are met, the vehicle computer prompts the user to start the upgrade program; otherwise, the user is not prompted to start the upgrade program, and the reason for not starting the upgrade program is displayed. The present invention can identify whether the environment around the vehicle is suitable for OTA upgrade through image scenes during the upgrade, thereby avoiding safety problems during the upgrade and ensuring the safety of the vehicle and personnel during the upgrade.

Description

Technical Field

[0001] The present invention relates to the field of vehicle safety control technology, and in particular to a vehicle OTA upgrade safety control method, system and storage medium based on image scene recognition. Background Art

[0002] The general process of vehicle OTA upgrade in the current industry is that the user downloads the upgrade package online on the vehicle computer; after the download is complete, the vehicle computer prompts the user to upgrade. If only the vehicle software is involved, the upgrade control module will detect conditions such as vehicle speed, power gear, and remaining space on the vehicle computer. If the conditions are met, the upgrade can be started. If the upgrade involves vehicle control software, such as gateways, BCM and other controllers, then the inverter gear must be in P gear, the handbrake is pulled, and the battery power and voltage are met.

[0003] For upgrades involving vehicle control software, although the above conditions need to be met before the upgrade, which ensures the safety of vehicles and personnel during the upgrade, this upgrade method still has risks and problems. For example, when a user drives to an intersection, if the traffic light is red, the user pulls up the handbrake while waiting, and other conditions such as the power gear are also met, then the conditions for upgrading the vehicle control software are met. If the upgrade starts at this time, the gateway enters the flashing gear and cannot switch to other gears. The vehicle cannot drive and will be stuck at the intersection, hindering traffic, and even causing traffic accidents due to the inability of the vehicle to drive, with disastrous consequences.

[0004] Therefore, it is very necessary to further ensure the safety of vehicle OTA upgrades and avoid the above risks. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the purpose of the present invention is to solve the problem of potential safety hazards in vehicle OTA upgrades in the prior art, and to provide a vehicle OTA upgrade safety control method, system and storage medium based on image scene recognition, which can identify whether the environment around the vehicle is suitable for OTA upgrades through image scenes during upgrades, thereby avoiding safety issues during upgrades and ensuring the safety of vehicles and personnel during upgrades.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] A vehicle OTA upgrade safety control method based on image scene recognition includes the following steps:

[0008] After the upgrade package is downloaded, the vehicle computer uses the camera to obtain the vehicle's external environment information;

[0009] The vehicle computer sends the acquired external environment information to the background for image scene recognition and obtains the background recognition results;

[0010] The vehicle computer obtains the vehicle's internal environment information;

[0011] The vehicle computer determines whether the upgrade conditions are met based on the background recognition results and internal environment information;

[0012] If the upgrade conditions are met, the vehicle computer prompts the user to start the upgrade program; otherwise, the user is not prompted to start the upgrade program, and the reason for not starting the upgrade program is displayed;

[0013] Among them, the external environment information includes image information around the vehicle; the internal environment information includes power gear, battery power, vehicle computer storage space, and transmission gear.

[0014] Furthermore, when the upgrade conditions are met, the vehicle computer prompts the user to start the upgrade program. After receiving the user's instruction to confirm the upgrade, the vehicle computer again calls the camera to obtain the vehicle's external environment information, and sends it to the background for image scene recognition, obtains the background recognition results, and again confirms that the upgrade conditions are met.

[0015] Furthermore, the vehicle computer sends the acquired external environment information to the background for image scene recognition, and obtains the background recognition result including:

[0016] The vehicle computer calls the open application programming interface of the background to send external environment information to the background;

[0017] The background uses a convolutional neural network model to perform image scene recognition on the external environment information to determine the scene the vehicle is currently in;

[0018] The background will return the recognition results to the vehicle computer.

[0019] Furthermore, the background returns the recognition result to the vehicle computer through inter-process communication.

[0020] Furthermore, the returned recognition result is JSON data about the scene; the fields of the data include location information and road information.

[0021] Furthermore, the following conditions are considered to meet the upgrade conditions:

[0022] The location information is not an intersection or a prohibited parking area; the road information is not a highway, a bridge, or a tunnel; and,

[0023] The vehicle speed is 0, the power gear is on, there is enough space left in the vehicle computer, the variable speed gear is in P gear, the handbrake is pulled, and the battery power and voltage are sufficient.

[0024] The present invention also provides a vehicle OTA upgrade safety control system based on image scene recognition, which is used in the method described, and includes a camera, a vehicle computer and a background; the vehicle computer includes an image processing module and an upgrade control module; wherein,

[0025] The camera is used to obtain the external environment information of the vehicle after receiving the call from the vehicle computer, and send the obtained external environment information to the image processing module of the vehicle computer;

[0026] The image processing module is used to process the image in the acquired external environment information, including image enhancement and image coding compression; it is also used to call the open application programming interface of the background to send the processed image to the background; it is also used to receive the recognition result returned after the background performs image scene recognition, and further transmit the recognition result to the upgrade control module through inter-process communication;

[0027] The upgrade control module is used to determine whether the upgrade conditions are met based on the background identification results and internal environment information;

[0028] The background is used to receive images processed by the image processing module; it is also used to use a convolutional neural network model to perform image scene recognition on the acquired images, determine the scene in which the vehicle is currently located, and return the scene information, i.e., the recognition result, to the image processing module.

[0029] Furthermore, the camera is connected to the vehicle computer via an LVDS line and mounted on a node on the vehicle computer; the camera data can be transmitted to the image processing module through the relevant underlying driver.

[0030] Furthermore, the background pre-stores a trained convolutional neural network model.

[0031] The present invention also provides a storage medium, which stores one or more programs. When the one or more programs are run by a processor, the steps of the vehicle OTA upgrade safety control method based on image scene recognition are executed.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] The method provided by the present invention not only confirms the internal environment of the vehicle before the vehicle is OTA upgraded, but also confirms the external environment of the vehicle through a camera, and recognizes the image of the external environment through a trained convolutional neural network model in the background, and then performs the vehicle OTA upgrade after confirming that the external environment meets the upgrade conditions. In this way, during the upgrade, it is possible to identify whether the environment around the vehicle is suitable for OTA upgrade through image scenes, thereby avoiding the occurrence of safety problems during the upgrade, and especially effectively avoiding safety problems that exist during the upgrade of vehicle control software, ensuring the safety of vehicles and personnel during the upgrade. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1This is a flow chart of a vehicle OTA upgrade safety control method based on image scene recognition in the present invention.

[0035] Figure 2 A schematic diagram of a vehicle OTA upgrade safety control system based on image scene recognition according to the present invention.

[0036] Figure 3 This is a flow chart of a vehicle OTA upgrade safety control method based on image scene recognition in Example 3 of the present invention. DETAILED DESCRIPTION

[0037] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0038] Embodiment 1

[0039] This embodiment discloses a vehicle OTA upgrade safety control method based on image scene recognition.

[0040] A vehicle OTA upgrade safety control method based on image scene recognition, see Figure 1 , including the following steps,

[0041] After the upgrade package is downloaded, the vehicle computer uses the camera to obtain the vehicle's external environment information.

[0042] The vehicle computer sends the acquired external environment information to the background for image scene recognition and obtains the background recognition results.

[0043] The vehicle computer obtains the vehicle's internal environment information.

[0044] The vehicle computer determines whether the upgrade conditions are met based on the background recognition results and internal environmental information.

[0045] If the upgrade conditions are met, the vehicle computer prompts the user to start the upgrade program. Otherwise, the user is not prompted to start the upgrade program, and the reason for not starting the upgrade program is displayed.

[0046] The external environment information includes image information around the vehicle, and the internal environment information includes power level, battery power, vehicle storage space, and transmission level.

[0047] In specific implementation, when the upgrade conditions are met, the vehicle computer prompts the user to start the upgrade program. After receiving the user's instruction to confirm the upgrade, the vehicle computer again calls the camera to obtain the vehicle's external environment information, and sends it to the background for image scene recognition, obtains the background recognition results, and again confirms that the upgrade conditions are met.

[0048] Furthermore, the vehicle computer sends the acquired external environment information to the background for image scene recognition, and obtains the background recognition result including:

[0049] The vehicle computer calls the open application programming interface of the background to send external environment information to the background.

[0050] The background uses a convolutional neural network model to perform image scene recognition on external environmental information to determine the current scene of the vehicle.

[0051] The background will return the recognition results to the vehicle computer.

[0052] During specific implementation, the background returns the recognition result to the vehicle computer through inter-process communication.

[0053] In specific implementation, the recognition result returned is JSON data about the scene, and the fields of the data include location information and road information.

[0054] Furthermore, the following conditions are considered to meet the upgrade conditions:

[0055] The location information is non-intersection and non-prohibited parking area; the road information is non-highway, non-bridge, and non-tunnel.

[0056] The vehicle speed is 0, the power gear is on, there is enough space left in the car computer, the variable speed gear is P, the handbrake is on, and the battery power and voltage are sufficient.

[0057] The method provided by the present invention not only confirms the internal environment of the vehicle before the vehicle is OTA upgraded, but also confirms the external environment of the vehicle through a camera, and recognizes the image of the external environment through a trained convolutional neural network model in the background, and then performs the vehicle OTA upgrade after confirming that the external environment meets the upgrade conditions. In this way, during the upgrade, it is possible to identify whether the environment around the vehicle is suitable for OTA upgrade through image scenes, thereby avoiding the occurrence of safety problems during the upgrade, and especially effectively avoiding safety problems that exist during the upgrade of vehicle control software, ensuring the safety of vehicles and personnel during the upgrade.

[0058] Embodiment 2

[0059] Based on the first embodiment, this embodiment discloses a vehicle OTA upgrade safety control system based on image scene recognition, which is used in the method.

[0060] The vehicle OTA upgrade safety control system based on image scene recognition, see Figure 2 , including a camera, a vehicle computer and a background. The vehicle computer includes an image processing module and an upgrade control module. Among them,

[0061] The camera is used to obtain the external environment information of the vehicle after receiving a call from the vehicle computer, and send the obtained external environment information to the image processing module of the vehicle computer.

[0062] The image processing module is used to process the image in the acquired external environment information, including image enhancement and image coding compression. It is also used to call the open application programming interface of the background to send the processed image to the background. It is also used to receive the recognition result returned by the background after image scene recognition, and further transmit the recognition result to the upgrade control module through inter-process communication.

[0063] The upgrade control module is used to determine whether the upgrade conditions are met according to the background identification results and internal environment information.

[0064] The background is used to receive the image processed by the image processing module and to use the convolutional neural network model to perform image scene recognition on the acquired image, determine the scene where the vehicle is currently located, and return the scene information, i.e., the recognition result, to the image processing module.

[0065] In specific implementation, the camera is connected to the vehicle computer via an LVDS line and mounted on a node on the vehicle computer, so that the camera data can be transmitted to the image processing module through the relevant underlying driver.

[0066] Furthermore, the background pre-stores a trained convolutional neural network model.

[0067] Due to the above-mentioned vehicle OTA upgrade safety control system based on image scene recognition, before the vehicle performs OTA upgrade, it is possible to confirm the vehicle's external environment through the camera, and recognize the image of the external environment through the trained convolutional neural network model in the background. After confirming that the external environment and internal environment meet the upgrade conditions, the vehicle OTA upgrade is performed, which can avoid the occurrence of safety problems during the upgrade, especially effectively avoid safety problems that exist during upgrades involving vehicle control software, and ensure the safety of vehicles and personnel during upgrades.

[0068] Embodiment 3

[0069] In order to further illustrate the actual effect of the vehicle OTA upgrade safety control based on image scene recognition of the present invention, this embodiment discloses a vehicle OTA upgrade safety control method based on image scene recognition implemented based on Embodiment 1 and Embodiment 2 and an application scenario.

[0070] See also Figure 3 On the vehicle computer side, after the upgrade package is downloaded, the vehicle computer will automatically call the camera to take pictures and perform scene recognition on the obtained images.

[0071] Specifically, when the user chooses to upgrade, the car computer will automatically call the camera to take pictures of the car's surroundings and transmit the captured images to the image processing module in the car computer. The module will process the images and transmit the image data to the background. The background will perform image scene recognition to determine what the scene is and return the result to the image processing module. The image processing module will then pass the obtained result information to the upgrade control module, which will determine whether the car computer can be upgraded based on the scene information and the vehicle's own conditions.

[0072] The camera is connected to the vehicle computer through an LVDS line and mounted on a node on the vehicle computer. The camera data can be transmitted to the image processing module of the Framework layer through the relevant underlying driver.

[0073] This module processes the image and then calls the relevant open API (Application Programming Interface) in the background to transmit the image data to the background tsp. The background uses the trained convolutional neural network model (CNN) to perform image scene recognition, thereby determining what the scene is and returning the result to the image processing module.

[0074] The result information can be further transmitted to the upgrade control module of the Application layer (i.e., the application layer) through AIDL (inter-process communication). The information can be JSON data about the scene, and the fields of the data include: location information, road information (which can be expanded later).

[0075] If the external scene meets the upgrade requirements, and the internal environment of the car also meets the upgrade conditions (power gear, battery power, storage space, transmission gear, etc.), then the user will be prompted to upgrade now. If any of them is not met, the user will not be prompted, and the upgrade conditions will be checked again after a period of time.

[0076] Specifically, the conditions for upgrading include: the location information must be non-intersection, non-prohibited parking area. The road information must be non-highway, non-bridge, and non-tunnel. The vehicle status must meet the requirements of 0 speed, power gear is ON, sufficient remaining space in the vehicle, variable gear is P, handbrake is on, and battery power and voltage are sufficient.

[0077] In addition, when the user actively triggers the upgrade, in order to ensure that the conditions are met in real time, the external environment will be photographed again and identified. If the scene meets the upgrade requirements and the vehicle's own status also meets the upgrade conditions, then the real process can be directly entered. If there are unsatisfied conditions, the upgrade will not be entered, and the user will be prompted for the reason why the upgrade cannot be performed. This effectively ensures the safety of the vehicle and personnel during the vehicle OTA upgrade.

[0078] Embodiment 4

[0079] This embodiment discloses a storage medium based on the first embodiment.

[0080] A storage medium storing one or more programs, wherein when the one or more programs are executed by a processor, the steps of the vehicle OTA upgrade safety control method based on image scene recognition are executed. The computer-readable storage medium may be a USB flash drive, a hard disk, or other device with storage function.

[0081] As described above, the reminder system of the present invention is not limited to the configuration described above, and other systems that can implement the embodiments of the present invention may fall within the scope of protection of the present invention.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit the technical solution. Those skilled in the art should understand that those modifications or equivalent substitutions of the technical solution of the present invention that do not depart from the purpose and scope of the technical solution should be included in the scope of the claims of the present invention.

Claims

1. A vehicle OTA upgrade safety control method based on image scene recognition, characterized in that: The following steps are included: After the upgrade package is downloaded, the vehicle computer uses the camera to obtain the vehicle's external environment information; The vehicle computer sends the acquired external environment information to the background for image scene recognition and obtains the background recognition results; The vehicle computer obtains the vehicle's internal environment information; The vehicle computer determines whether the upgrade conditions are met based on the background recognition results and internal environment information; The following conditions are considered to meet the upgrade conditions: The location information is not an intersection or a prohibited parking area; the road information is not a highway, a bridge, or a tunnel; and, The vehicle speed is 0, the power gear is in the ON gear, there is enough space left in the vehicle computer, the inverter gear is in the P gear, the parking brake is pulled, and the battery power and voltage are sufficient; If the upgrade conditions are met, the vehicle computer prompts the user to start the upgrade program; otherwise, the user is not prompted to start the upgrade program, and the reason for not starting the upgrade program is displayed; Among them, the external environment information includes image information around the vehicle; the internal environment information includes power gear, battery power, vehicle computer storage space and transmission gear.

2. The vehicle OTA upgrade safety control method based on image scene recognition according to claim 1 is characterized in that: When the upgrade conditions are met, the vehicle computer prompts the user to start the upgrade program. After receiving the user's instruction to confirm the upgrade, the vehicle computer again calls the camera to obtain the vehicle's external environment information, and sends it to the background for image scene recognition, obtains the background recognition results, and again confirms that the upgrade conditions are met.

3. The vehicle OTA upgrade safety control method based on image scene recognition according to claim 1 is characterized in that: The vehicle computer sends the acquired external environment information to the background for image scene recognition, and obtains the background recognition result including: The vehicle computer calls the open application programming interface of the background to send external environment information to the background; The background uses a convolutional neural network model to perform image scene recognition on the external environment information to determine the scene the vehicle is currently in; The background will return the recognition results to the vehicle computer.

4. The vehicle OTA upgrade safety control method based on image scene recognition according to claim 3 is characterized in that: The background returns the recognition results to the vehicle computer through inter-process communication.

5. The vehicle OTA upgrade safety control method based on image scene recognition according to claim 3 or 4 is characterized in that: The returned recognition result is JSON data about the scene; the fields of the data include location information and road information.

6. A vehicle OTA upgrade safety control system based on image scene recognition, characterized in that: Used in the method according to any one of claims 1 to 5, comprising a camera, a vehicle computer and a background; the vehicle computer comprises an image processing module and an upgrade control module; wherein, The camera is used to obtain the external environment information of the vehicle after receiving the call from the vehicle computer, and send the obtained external environment information to the image processing module of the vehicle computer; The image processing module is used to process the image in the acquired external environment information, including image enhancement and image coding compression; it is also used to call the open application programming interface of the background to send the processed image to the background; it is also used to receive the recognition result returned after the background performs image scene recognition, and further transmit the recognition result to the upgrade control module through inter-process communication; The upgrade control module is used to determine whether the upgrade conditions are met based on the background identification results and internal environment information; The background is used to receive images processed by the image processing module; it is also used to use a convolutional neural network model to perform image scene recognition on the acquired images, determine the scene in which the vehicle is currently located, and return the scene information, i.e., the recognition result, to the image processing module.

7. The vehicle OTA upgrade safety control system based on image scene recognition according to claim 6 is characterized in that: The camera is connected to the vehicle computer via an LVDS line and mounted on a node on the vehicle computer; the camera data can be transmitted to the image processing module through the relevant underlying driver.

8. The vehicle OTA upgrade safety control system based on image scene recognition according to claim 6, characterized in that: The background pre-stores a trained convolutional neural network model.

9. A storage medium, characterized in that: The storage medium stores one or more programs, and when the one or more programs are executed by the processor, the steps of the vehicle OTA upgrade safety control method based on image scene recognition as described in any one of claims 1 to 5 are executed.

Images