Fault processing method, vehicle and storage medium

Abstract

The invention provides a fault processing method, a vehicle and a storage medium, and relates to the technical field of vehicles. The method comprises the following steps: detecting states of a target camera and target software; wherein the target camera is a vehicle camera; the target software refers to software which needs to use an image collected by the target camera; under the condition that the target camera fault or the target software fault is detected, determining an external broadcasting strategy of fault information according to a privacy switch state; wherein the privacy switch state is used for indicating whether to transmit user privacy information to the outside or not. According to the technical scheme, the problem that in the prior art, when the vehicle camera or related software using the vehicle camera breaks down and the fault information is broadcasted to the outside, user privacy information is prone to being leaked can be solved.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a fault handling method, a vehicle, and a storage medium. Background Art

[0002] With the development of vehicle technology, vehicle cameras have gradually become an important component for improving driving safety and intelligence, and are widely used in functional modules such as driver status monitoring, passenger behavior analysis, and personalized services.

[0003] However, when a vehicle camera or related software using the vehicle camera malfunctions, the vehicle may need to broadcast the malfunction information externally, such as sending it to a remote server or after-sales service center for prompt repair. However, the malfunction information broadcast externally may contain private information such as the facial features and voice characteristics of passengers inside or outside the vehicle, which may easily lead to the leakage of user privacy information. Summary of the Invention

[0004] Based on the above-mentioned defects and shortcomings of the existing technology, the present application proposes a fault handling method, a vehicle and a storage medium, which can solve the problem in the existing technology that user privacy information is easily leaked when the vehicle camera or the related software using the vehicle camera fails and the fault information is broadcast to the outside.

[0005] According to a first aspect of the present application, a fault handling method is provided, the method comprising:

[0006] Detecting the status of a target camera and target software; wherein the target camera refers to a vehicle camera; and the target software refers to software that needs to use data collected by the target camera;

[0007] When a failure of the target camera or the target software is detected, a policy for externally broadcasting the failure information is determined according to the privacy switch status; wherein the privacy switch status is used to indicate whether to transmit user privacy information externally.

[0008] In some optional embodiments, determining a strategy for reporting fault information to the outside world based on the privacy switch state includes:

[0009] When the privacy switch is in the on state, it is determined that the fault information is not broadcast externally.

[0010] In some optional embodiments, when the target software includes driver monitoring system software, upon detecting a target camera failure or a target software failure, determining a strategy for externally broadcasting the failure information based on the privacy switch state includes:

[0011] When a target camera failure or a target software failure is detected, a strategy for externally broadcasting the fault information is determined based on the privacy switch status and the switch status of the driver monitoring system.

[0012] In some optional embodiments, when a target camera failure or a target software failure is detected, a strategy for externally broadcasting the failure information is determined based on the privacy switch status and the switch status of the driver monitoring system, including:

[0013] When a failure of the target camera or the target software is detected, if the privacy switch is in the on state, determining not to broadcast the failure information externally;

[0014] In the event of a first camera failure or a driver monitoring system software failure being detected, if the privacy switch is in the off state and the driver monitoring system switch is in the on state, determining to broadcast a first fault message externally; wherein the first camera is a target camera used by the driver monitoring system, and the first fault message is used to instruct the driver monitoring system to stop operating;

[0015] In the case of detecting a failure of the target camera, if the privacy switch and the switch of the driver monitoring system are both in the off state, determining to broadcast a second failure message externally; wherein the second failure message is used to indicate the failure of the target camera;

[0016] When a target software failure is detected, if both the privacy switch and the switch of the driver monitoring system are in the off state, it is determined that the failure information will not be broadcast externally.

[0017] In some optional embodiments, the second fault information is further used to indicate that the target software using the faulty target camera cannot be used normally.

[0018] In some optional embodiments, when a target camera failure or a target software failure is detected, a strategy for externally broadcasting the fault information is determined based on the privacy switch status and the switch status of the driver monitoring system, including:

[0019] In the case where a fault in the target software is detected, if the switch states of the privacy switch and the driver monitoring system are both in the off state, determining the degree of impact of the target software on vehicle driving;

[0020] When the impact of the target software on vehicle driving is greater than a preset value, determining to broadcast third fault information externally; wherein the third fault information is used to indicate the target software fault;

[0021] If the impact of the target software on vehicle driving is less than or equal to a preset value, it is determined not to broadcast the fault information externally.

[0022] In some optional embodiments, determining a strategy for reporting fault information to the outside world based on the privacy switch state includes:

[0023] When the privacy switch is in the on state, the user privacy information in the fault information is removed, and the fault information with the user privacy information removed is broadcasted externally.

[0024] In some optional embodiments, detecting the status of the target camera and the target software includes:

[0025] In the case where it is detected that the target software cannot work normally, detecting whether a target camera used by the target software is faulty;

[0026] When the target camera used by the target software is not faulty, it is determined that the target software itself is faulty.

[0027] According to a second aspect of the present application, a fault handling device is provided, the device comprising:

[0028] A detection module, configured to detect the status of a target camera and target software; wherein the target camera is a vehicle camera; and the target software is software that requires data collected by the target camera;

[0029] A processing module is used to determine an external broadcasting strategy for the fault information according to the privacy switch state when a fault of the target camera or the target software is detected; wherein the privacy switch state is used to indicate whether to transmit user privacy information externally.

[0030] According to a third aspect of the present application, there is provided an electronic device, comprising: a memory and a processor;

[0031] The memory is connected to the processor and is used to store programs;

[0032] The processor is used to implement the fault handling method as described in the first aspect by running the program in the memory.

[0033] According to a fourth aspect of the present application, a vehicle is provided, comprising the electronic device as described in the third aspect, wherein the vehicle implements the fault handling method as described in the first aspect through the electronic device.

[0034] According to a fifth aspect of the present application, a storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the fault handling method as described in the first aspect is implemented.

[0035] According to a sixth aspect of the present application, a computer program product or a computer program is provided, wherein the computer program product includes the computer program, and when a processor executes the computer program, the steps in the fault handling method described in the first aspect are implemented.

[0036] The technical solution provided in this application provides users with privacy control by setting a privacy switch. When a vehicle camera malfunction and / or a software malfunction related to the vehicle camera is detected, the policy for externally reporting the malfunction information can be determined based on the privacy switch status. When the privacy switch is on, at least no user private information is transmitted externally, thereby preventing the leakage of user private information and improving the security of user privacy. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] By reading the detailed description of the embodiments below, the advantages and benefits of various embodiments will become clear to those skilled in the art. In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only embodiments of the present application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative work.

[0038] Figure 1 A flowchart of a fault handling method provided in an embodiment of the present application;

[0039] Figure 2 A schematic diagram of the process of distracted driving behavior monitoring provided in an embodiment of the present application;

[0040] Figure 3 A schematic diagram of a fault information external broadcasting strategy provided in an embodiment of the present application;

[0041] Figure 4 A block diagram of a fault handling device provided in an embodiment of the present application;

[0042] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0043] Figure 6 A schematic structural diagram of a vehicle provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0045] Application Overview

[0046] With the development of vehicle technology, vehicle cameras have gradually become an important component for improving driving safety and intelligence. They are widely used in functional modules such as driver status monitoring (such as fatigue driving warning and distraction detection), passenger behavior analysis (such as gesture control and child left behind detection), personalized services (such as facial recognition unlocking and emotion recognition to adjust the cabin environment), and vehicle surrounding environment perception. For example, the Driver Monitoring System (DMS) can track the driver's eye state and head posture through the vehicle camera to determine whether the driver has abnormal behavior such as closing their eyes or lowering their head; the Occupancy Monitoring System (OMS) can monitor the presence and status of passengers in the vehicle in real time through the vehicle camera; the facial recognition system can recognize the driver's facial features through the vehicle camera to start the vehicle, etc.; and the perception module can perceive the vehicle's surrounding environment through the vehicle camera.

[0047] However, when a vehicle camera malfunctions (e.g., circuit disconnection, component damage, etc.) or related software that requires the vehicle camera fails (e.g., algorithm crash, data frame loss, etc.), the vehicle may need to send fault information via the onboard communication module to the vehicle manufacturer's terminal, service provider's terminal, or the vehicle owner's terminal, so that measures can be taken quickly to repair or adjust the fault and ensure the normal operation of the system and driving safety. For example, when the system detects a camera malfunction or related software failure, timely notification can help the vehicle owner arrange for repair service and avoid safety hazards caused by system failure. In addition, for remote monitoring services, understanding the health of the vehicle and its systems is a key part of ensuring operational efficiency and safety.

[0048] However, such information may include detailed diagnostic data and logs, which may directly or indirectly reveal the user's personal information. For example, the vehicle system records the images or video clips captured by the camera at a specific time point and saves them as part of the fault information. These image data may contain the facial features or other identifiable information of the driver or other personnel. In the process of externally transmitting the fault, the user's biometrics, behavioral habits and other private information may be exposed to third-party operation and maintenance platforms or network environments, posing a privacy risk. In addition, the false triggering of the reporting mechanism under fault conditions may further expand the scope of privacy leakage, such as when the software mistakenly marks a normal scene as an abnormal event and uploads it to the cloud.

[0049] As users' privacy awareness increases, the use of vehicle cameras has caused users to worry about the leakage of personal data. Especially when it comes to facial information collection, users hope to have greater privacy control, so there is an urgent need for a solution that can better protect user privacy.

[0050] To this end, an embodiment of the present application provides a fault handling solution, which provides users with privacy control by setting a privacy switch. When a vehicle camera fault and / or a software fault related to the vehicle camera is detected, the external broadcasting strategy of the fault information can be determined according to the state of the privacy switch. When the privacy switch is in the on state, at least the user's privacy information is not transmitted externally, thereby avoiding the leakage of the user's privacy information. For example, when the privacy switch is in the off state, the fault information is broadcast externally normally; when the privacy switch is in the on state, the fault information is not broadcast externally, thereby avoiding the leakage of the user's privacy information. Of course, this is just an example, and it is not the only implementation method.

[0051] For details about the troubleshooting solution provided in the embodiments of this application, please see below.

[0052] Exemplary Methods

[0053] An embodiment of the present application provides a fault handling method, which is applied to a vehicle equipped with a vehicle camera, wherein the vehicle camera is used to provide the collected data to the relevant software in the vehicle, and the data described here may include at least one of image data and sound data.

[0054] The vehicle camera can be installed on the vehicle before or after it leaves the factory. The vehicle camera's installation location can be determined to facilitate the capture of images inside or outside the vehicle. For example, it can be installed on the A-pillar (where the front door meets the roof) to capture the driver's facial features for fatigue detection, distraction detection, and identity authentication. It can also be installed on the B-pillar (between the front and rear doors) to observe the behavior of the front passenger and rear passengers for child detection and seat belt reminders. It can also be installed on the base of the rearview mirror, leveraging the rearview mirror's field of view to cover the entire vehicle cabin and capture front and rear passenger actions from a bird's-eye view (such as gesture control and left-behind object detection). It can also be installed on the exterior of the vehicle to capture images of the vehicle's surroundings. The type of vehicle camera can also be set according to actual needs, such as visible light camera, infrared camera, binocular / multi-lens camera, etc.

[0055] The following embodiments describe the method in detail. The following embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0056] Among them, the executor of the fault handling method provided in the embodiment of the present application can be a vehicle controller, which can include but is not limited to one of the following: a vehicle controller, a cockpit domain controller, an intelligent driving domain controller, etc.

[0057] like Figure 1 As shown, the method may include steps 101 to 102, as described below.

[0058] Step 101: Detect the status of the target camera and target software.

[0059] The target camera mentioned here refers to a vehicle camera used to capture images inside or outside the vehicle, and the data it captures is used for software in the vehicle.

[0060] The target software mentioned here refers to software that requires data collected by the target camera, such as driver monitoring system software, passenger monitoring system software, face recognition system software, perception module, etc. The data mentioned here may include at least one of the following: image data, sound data.

[0061] Among them, the number of target cameras and target software can be at least one. When the number of target cameras and target software is one, the two are in a one-to-one correspondence, that is, the data collected by the target camera is used for the target software. When the number of target cameras is one and the number of target software is multiple, the two are in a one-to-many relationship, that is, the data collected by the target camera is used for multiple target software. When the number of target cameras is multiple and the number of target software is multiple, one of the target cameras can provide collected data for one of the target software; one of the target cameras can provide collected data for multiple target software; or multiple target cameras can provide collected data for one of the target software. The correspondence between the target cameras and the target software is pre-set, that is, it is known which target software uses which target cameras.

[0062] Among them, the status of the target camera and the target software may include but is not limited to: normal status, fault status, etc. In the embodiment of the present application, the status of the target camera and the target software can be detected in real time or at a fixed time after the vehicle is started, so as to obtain the status of the two in a timely manner; the status of the target camera and the target software can also be detected in real time or at a fixed time during the use of the target camera and the target software to reduce invalid detection. For example, the driver monitoring system software is mainly used to monitor the status of the driver while the vehicle is driving. Therefore, for the driver monitoring system software and the vehicle camera used by it, the status of the two can be detected in real time or at a fixed time after the vehicle is started and the vehicle is in a driving state; for another example, the face recognition system software is mainly used to start the vehicle through identity authentication. After the vehicle is started, the face recognition system software may no longer be used. Therefore, for the face recognition system software and the vehicle camera used by it, the status of the two can be detected in real time or at a fixed time after the vehicle is unlocked and before the vehicle is started.

[0063] Step 102: When a target camera failure and / or a target software failure is detected, a strategy for externally broadcasting the failure information is determined based on the privacy switch status.

[0064] The privacy switch described here is a switch set for the user in an embodiment of the present application, which is used to indicate whether the user's privacy information is transmitted externally. The user has control over the privacy switch, that is, the user can turn on the privacy switch or turn off the privacy switch. The privacy switch can be a physical switch or a virtual switch. In the case where the privacy switch is a physical switch, the position of the privacy switch can be set according to actual needs, such as near the instrument panel, on the steering wheel, and other locations that are easy to view and operate. When the privacy switch is a virtual switch, it can be set on the main page of the central control screen, or it can be deployed in the settings menu page, and it can also be set on other pages according to actual needs.

[0065] In an embodiment of the present application, when a target camera failure and / or target software failure is detected, a policy for externally broadcasting the failure information can be determined based on the state of the privacy switch, and the failure information can be broadcasted normally. When the privacy switch is in the on state, at least no user private information is transmitted externally, thereby preventing leakage of user private information.

[0066] In some embodiments, when the privacy switch is in the on state, fault information may not be broadcast externally, thereby better avoiding the leakage of user privacy information during the information broadcast process and improving the security of user privacy information.

[0067] In other embodiments, when the privacy switch is in the on state, the user privacy information in the fault information can also be removed, and then the fault information with the user privacy information removed can be broadcast externally. In this way, while avoiding the leakage of user privacy information during the information broadcast process, the fault information can also be notified to relevant departments or personnel so that the fault problem can be solved in a timely manner.

[0068] Optionally, features of user privacy information (such as facial features, voice features, etc.) can be collected in advance, and then based on the features of the collected user privacy information, the user privacy information can be matched and searched in the fault information to remove it and achieve a desensitization effect; or an artificial intelligence model can be established and trained in advance, and the fault information can be input into the trained artificial intelligence model, so that the artificial intelligence model can identify the user privacy information in the fault information and remove it to achieve a desensitization effect. The specific desensitization method can be set according to actual needs, and the embodiments of this application do not specifically limit this.

[0069] Statistics show that subjective factors such as fatigue and distracted driving are the primary causes of traffic accidents. Driver monitoring systems, based on artificial intelligence and sensor technology, monitor the driver's status, behavior, and driving environment in real time, identifying driver fatigue and distracted driving behaviors, and providing driver alerts, thereby effectively reducing the risk of traffic accidents caused by these behaviors. In some embodiments, the driver monitoring system can provide different levels of alerts based on the duration of distracted driving when monitoring distracted driving.

[0070] like Figure 2As shown, when the driver monitoring system detects that the driver's eyes are off the road ahead (e.g., due to checking the left or right exterior mirrors, the instrument panel, or talking to the front passenger) for a duration less than or equal to a first duration (e.g., 3 seconds, 4 seconds, etc., which can be set according to actual needs), the driver monitoring system can determine that the distracted driving behavior is prolonged. At this time, the driver monitoring system can control the output of an audible reminder (e.g., "You are distracted driving, please drive safely") and / or a visual reminder (e.g., a warning message displayed on the central control screen) to remind the driver to refocus on the road ahead. This instant feedback mechanism can quickly capture the driver's attention and effectively prevent safety hazards caused by prolonged distraction.

[0071] like Figure 2 As shown, when the driver monitoring system detects that the driver's eyes are away from the road ahead for a duration greater than or equal to a second duration (e.g., 2 seconds) and less than the first duration, the driver monitoring system can determine that the distracted driving behavior is a short-term distraction. The driver monitoring system can control the steering wheel to slightly vibrate as a reminder to guide the driver's attention to the road ahead. This method does not suddenly interrupt the driver's normal operating process, but rather gently prompts them to pay more attention to the road conditions. It is particularly suitable for addressing frequent but brief distractions.

[0072] Alternatively, if the driver's gaze briefly shifts away from the road ahead while observing the rearview mirror or checking the instrument panel, and if they promptly return to their focus after completing their observation and checking, a distraction reminder may not be issued, as this is normal driving behavior. Frequent distraction reminders, on the other hand, may annoy the driver and affect their driving experience. Therefore, for short-term distractions, a determination may be made based on the frequency and / or cumulative duration of such distractions over a period of time. For example, if the number of short-term distractions within a third time period (e.g., 30 seconds, which can be set based on actual needs) is greater than or equal to a threshold (e.g., 5 or 6 times, which can be set based on actual needs), the distraction is considered frequent and affects driving safety. In this case, the driver monitoring system may control the steering wheel to vibrate slightly to alert the driver. For another example, if the cumulative duration of the driver's short-term distraction within the third time period is greater than or equal to the fourth time period (the fourth time period is less than the third time period, such as 10 seconds, which can be set according to actual needs), and the time the driver's eyes are fixed on the road ahead each time does not exceed the fifth time period (such as 2 seconds, which can be set according to actual needs), then the driver's distraction is considered to affect driving safety, and the driver monitoring system can control the steering wheel to vibrate slightly to guide the driver to pay attention to the road ahead.

[0073] It should be noted that the driver monitoring system may detect the cause of the line of sight deviation, or may not detect the cause of the line of sight deviation but only detect whether the line of sight is deviated. The specific setting can be made according to actual needs.

[0074] At present, the driver monitoring system has become an important part of the intelligent vehicle safety system, and the driver monitoring system requires the use of a vehicle camera. Therefore, the target software described in the embodiment of the present application may include the driver monitoring system software, and the target camera described in the embodiment of the present application may include the camera used by the driver monitoring system software (hereinafter referred to as the first camera).

[0075] Since the driver monitoring system is an important system that affects vehicle safety, its failure will seriously affect the safety of vehicle driving. Therefore, in some embodiments, when a target camera failure and / or target software failure is detected, the strategy for externally broadcasting the fault information can also be determined based on the privacy switch status and the switch status of the driver monitoring system. In this way, while paying attention to user privacy information, it is also possible to pay attention to the safety of vehicle driving.

[0076] The driver monitoring system's on / off status can be controlled by the vehicle. For example, for some vehicles, the driver monitoring system automatically turns on when the ignition is turned on or the power is applied, and automatically turns off when the vehicle is turned off or the power is lost. For other vehicles, the driver monitoring system automatically turns on when the vehicle is in gear (such as D or R), and automatically turns off when the vehicle is turned off. Of course, the driver monitoring system's on / off status can also be controlled by the user. For example, the user can enter the settings menu on the central control screen and select to turn the driver monitoring system on or off; or turn the driver monitoring system on or off through voice commands.

[0077] Among them, the driver monitoring system can include software parts (such as driver behavior analysis algorithms, etc.) and hardware parts (such as in-vehicle cameras, seat sensors, steering wheel tactile sensors, etc.).

[0078] like Figure 3 As shown, the embodiment of the present application can set different external broadcast strategies for camera failure and software failure respectively. Figure 3 This is further described.

[0079] Alternatively, as Figure 3 As shown, when a target camera failure is detected, the strategy for externally broadcasting the failure information is determined based on the privacy switch status and the switch status of the driver monitoring system, which can be described as follows:

[0080] A1. If a target camera fault is detected and the privacy switch is on and the driver monitoring system switch is off, the fault information will not be broadcast externally.

[0081] B1. If a target camera fault is detected and both the privacy switch and the driver monitoring system switch are on, the fault information will not be broadcast externally.

[0082] C1. When the privacy switch is off and the driver monitoring system is on, if a first camera fault is detected, a first fault message is broadcast. The first camera is the target camera used by the driver monitoring system, and the first fault message instructs the driver monitoring system to stop operating.

[0083] D1. When a target camera fault is detected, if both the privacy switch and the driver monitoring system switch are in the off state, a second fault message is broadcast externally. The second fault message is used to indicate a target camera fault.

[0084] From the fault information external broadcasting strategies described in A1 and B1 above, it can be seen that as long as the privacy switch is in the on state, no fault information will be broadcast externally regardless of whether the driver monitoring system switch is in the on state or the off state, thereby avoiding the leakage of user privacy information and improving the security of user privacy information.

[0085] As can be seen from the fault information reporting strategy described in C1 above, if the privacy switch is off and all DMS switches are on, and the target camera used by the DMS (i.e., the first camera) fails, a fault message instructing the DMS to cease operation will be broadcast. This is because the vehicle system's primary task is to ensure that the driver (or vehicle owner) and relevant departments are aware of the failure of the DMS function directly related to driving safety. Although the first camera may be used for other functions besides the DMS, such as facial recognition and other personalized services (such as seat position memory and emotion recognition), in this case, prioritizing notification of the DMS function's unavailability allows the driver and relevant departments to take timely measures to ensure driving safety. The impact of the first camera's use for other functions is relatively minor and does not directly affect the current driving state. Therefore, the vehicle system can focus on reporting fault information that has a significant impact on driving safety.

[0086] It is understood that, when the privacy switch is off and all the switches of the driver monitoring system are on, if the first camera malfunctions and the first camera is also used for other target software, then the first fault message indicating that the driver monitoring system has stopped working may be broadcasted externally at the same time as the fault message indicating that the other target software has failed. Similarly, when the privacy switch is off and all the switches of the driver monitoring system are on, if the first camera malfunctions and other target cameras also malfunction, then the first fault message indicating that the driver monitoring system has stopped working may be broadcasted externally at the same time as the fault message related to the other target cameras.

[0087] Regarding the fault information external broadcasting strategy described in D1 above, when a target camera fault is detected, if the privacy switch and the switch of the driver monitoring system are both in the off state, a fault message indicating the target camera fault can be broadcast externally. The functional scope of the target camera may be larger than that of the driver monitoring system. It can not only be used for driver monitoring, but also support a variety of other functions, such as face recognition, emotion detection, gesture recognition, etc. Therefore, even if the driver monitoring system is turned off, the target camera may still serve other key functions (such as face recognition, cockpit monitoring, etc.). If the target camera fails, these functions that rely on the target camera will not work properly, affecting the user experience and certain safety features of the vehicle. Therefore, it is necessary to broadcast the target camera fault information externally so that relevant personnel and relevant departments can take corresponding measures to solve the fault in a timely manner and improve vehicle safety and user experience.

[0088] Optionally, the second fault information described in D1 above can also be used to indicate that the target software using the faulty target camera cannot be used normally, thereby notifying the user that relevant functions are restricted or unavailable, prompting the user to arrange repairs as soon as possible, restore all camera-dependent functions, and ensure the integrity and reliability of the overall system.

[0089] Among them, for the aforementioned D1, in addition to broadcasting the second fault information for indicating the fault of the target camera, a fault information for instructing the target software using the faulty target camera to stop working can also be broadcasted.

[0090] Optionally, when a target software failure is detected, the strategy for broadcasting the failure information externally may be determined based on the privacy switch status and the driver monitoring system switch status as follows:

[0091] A2. When a target software failure is detected, if the privacy switch is on and the driver monitoring system switches are off, the failure information will not be broadcast externally.

[0092] B2. If a target software failure is detected and both the privacy switch and the driver monitoring system switch are on, the failure information will not be broadcast externally.

[0093] C2. When the privacy switch is off and the driver monitoring system switch is on, if a driver monitoring system software failure is detected, a third failure message is broadcast externally. The third failure message is used to instruct the driver monitoring system to stop working.

[0094] D2. When a target software failure is detected, if both the privacy switch and the driver monitoring system are turned off, no fault information will be broadcast externally.

[0095] From the fault information external broadcasting strategies described in A2 and B2 above, it can be seen that as long as the privacy switch is in the on state, no fault information will be broadcast externally regardless of whether the driver monitoring system switch is in the on state or the off state, thereby avoiding the leakage of user privacy information and improving the security of user privacy information.

[0096] It can be seen from the fault information external broadcasting strategy described in C2 above that when the privacy switch is in the off state, if the driver monitoring system software fails, it is necessary to broadcast the first fault information to instruct the driver monitoring system to stop working, so that relevant personnel and departments know that the driver monitoring system function directly related to driving safety has failed, and thus take timely measures to ensure driving safety.

[0097] Regarding the fault information reporting strategy described in D2 above, if both the privacy switch and the driver monitoring system are off, the faulty software can be excluded from the driver monitoring system software. To avoid unnecessary interference, the fault information may not be reported externally. Of course, it is understood that the relevant fault information may also be reported externally.

[0098] When both the privacy switch and the driver monitoring system are off, it is also possible to determine whether to broadcast the fault information based on the impact of the faulty target software on vehicle driving. The specific implementation method is as follows:

[0099] When a target software failure is detected, if the privacy switch and the driver monitoring system are both in the off state, the degree of impact of the target software on driving is determined; if the degree of impact of the target software on driving is greater than a preset value, a third fault information can be broadcast externally; wherein, the third fault information is used to indicate a target software failure; if the degree of impact of the target software on driving is less than or equal to the preset value, the fault information is not broadcast externally.

[0100] Suppose a user is driving a vehicle with both the privacy switch and the driver monitoring system switched off. The vehicle system then detects a target software failure, such as a failure in the facial recognition system's software module. While different from a DMS system, facial recognition systems also rely on cameras to automatically identify the driver upon boarding the vehicle and automatically adjust settings like the seat and rearview mirrors based on the driver's identity. The system can then determine the importance of the facial recognition system software to vehicle driving. If this importance is greater than a preset value, the relevant fault information is broadcast. Conversely, if the importance is less than or equal to the preset value, the relevant fault information is not broadcast.

[0101] Optionally, importance levels can be set based on the different functions involved in the target software. For example, if a software module in a facial recognition system fails, the vehicle system can determine which functions are affected by the software failure and then determine the importance level of each function. If the importance level of any function exceeds a preset value, the relevant failure information will be announced. Conversely, if the importance levels of all functions are less than or equal to the preset value, the relevant failure information will not be announced.

[0102] In the embodiment of the present application, whether to broadcast fault information externally is determined based on the degree of impact of the faulty target software on vehicle driving. This can not only report important faults in a timely manner, but also reduce interference from irrelevant fault reports.

[0103] It is understandable that the degree of influence of different target software on vehicle driving and preset values ​​for comparison can be preset.

[0104] In some embodiments, when it is detected that the target software is not functioning properly (e.g., a target function has failed), the target camera used by the target software may be checked for malfunction. If the target camera used by the target software is not malfunctioning, it may be determined that the target software itself is malfunctioning. The target function herein refers to a function that requires the use of images captured by the target camera to implement.

[0105] Since the target camera is the primary data source for the target software, if the target camera itself experiences an anomaly (such as occlusion or disconnection), the target software will not function effectively even if it is functioning normally. Therefore, first detecting whether the target camera is faulty helps quickly locate the source of the problem. Furthermore, a functional failure of the target software caused by a target camera anomaly may be misidentified as a fault in the target software itself, resulting in incorrect processing logic or user prompts. Therefore, first determining whether the target camera is faulty also helps distinguish the fault level and avoid misjudgments.

[0106] Among them, since the failure of the target software itself will not cause the target camera used by it to fail, if a failure of the target camera is detected, it can be determined that there is a failure in the target camera itself.

[0107] Among them, if it is detected that the target software is not working properly, if neither the target software nor the target camera has failed, it is also possible to determine whether the target camera is blocked. Because if the target camera is blocked, the target camera cannot capture valid images, which will also cause the target software to not work properly. Optionally, it is possible to determine whether the target camera is blocked by checking whether there are large areas of obstructions (such as black or solid color areas) in the image captured by the target camera. Computer vision algorithms can also be used to identify abnormal occlusion patterns in the image, such as the sudden appearance of a large area of ​​blur or an unchanging background. Some target cameras are equipped with occlusion detection sensors, so the occlusion status can be fed back through the occlusion detection sensors.

[0108] It should be noted that the target software failure required in the embodiments of the present application can refer to a failure of the target software itself or a failure caused by the target camera. In order to accurately determine the reporting strategy of the fault information, it is preferred that the target software failure refers to a failure of the target software itself. Similarly, the target camera failure refers to a failure of the target camera itself.

[0109] It's also important to note that in the event of a malfunction in the target camera or software, a fault alert can be provided within the vehicle to inform occupants. For example, in the event of a facial recognition system malfunction, a targeted notification can be provided, such as "Face recognition is temporarily unavailable, please adjust the seat manually," rather than simply reporting a "Face module crash" error.

[0110] In summary, in the embodiments of the present application, a privacy switch is set to provide users with privacy control. When a vehicle camera malfunction or a software malfunction related to the vehicle camera is detected, a policy for externally reporting the malfunction information is determined based on the privacy switch status. When the privacy switch is on, at least no user private information is transmitted externally, thereby preventing the leakage of user private information and improving the security of user privacy.

[0111] Exemplary devices

[0112] Correspondingly, an embodiment of the present application also provides a fault handling device, which is applied to a vehicle equipped with a vehicle camera, and the vehicle camera is used to provide the collected data to the relevant software in the vehicle. The data mentioned here may include at least one of image data and sound data.

[0113] like Figure 4As shown, the device may include:

[0114] The detection module 401 is used to detect the status of the target camera and the target software; wherein the target camera refers to the vehicle camera; the target software refers to the software that needs to use the data collected by the target camera.

[0115] The processing module 402 is used to determine the external broadcasting strategy of the fault information according to the privacy switch status when the target camera fault and / or the target software fault is detected; wherein the privacy switch status is used to indicate whether the user's private information is transmitted externally.

[0116] In some optional embodiments, the processing module 402 may be specifically configured to determine not to broadcast fault information externally when the privacy switch is in an on state.

[0117] In some optional embodiments, when the target software includes driver monitoring system software, the processing module 402 may include:

[0118] The processing unit is used to determine a strategy for externally broadcasting fault information based on the privacy switch status and the switch status of the driver monitoring system when a fault of the target camera or the target software is detected.

[0119] In some optional embodiments, the processing unit may be specifically configured to:

[0120] In the event that a failure of the target camera or the target software is detected, if the privacy switch is in the on state, no failure information will be broadcast externally;

[0121] In the event of a first camera failure or a driver monitoring system software failure being detected, if the privacy switch is in the off state and the driver monitoring system switch is in the on state, a first fault message is broadcasted externally; wherein the first camera is a target camera used by the driver monitoring system, and the first fault message is used to instruct the driver monitoring system to stop operating;

[0122] In the event that a target camera failure is detected, if both the privacy switch and the driver monitoring system switch are in the off state, a second fault message is broadcast externally; wherein the second fault message is used to indicate that the target camera has failed;

[0123] When a target software failure is detected, if both the privacy switch and the driver monitoring system switch are in the off state, no failure information will be broadcast externally.

[0124] In some optional embodiments, the processing unit may further be specifically configured to:

[0125] In the case where a fault in the target software is detected, if the switch states of the privacy switch and the driver monitoring system are both in the off state, determining the degree of impact of the target software on driving;

[0126] When the impact of the target software on driving is greater than a preset value, a third fault message is broadcast externally; wherein the third fault message is used to indicate the target software fault;

[0127] If the impact of the target software on driving is less than or equal to a preset value, no fault information will be broadcast externally.

[0128] In some optional embodiments, the processing module 402 may be specifically configured to:

[0129] When the privacy switch is in the on state, the user privacy information in the fault information is removed, and the fault information in which the user privacy information is removed is broadcasted externally; wherein the user privacy information includes the user's face image.

[0130] In some optional embodiments, the detection module 401 may be specifically configured to:

[0131] In the case where the target software is detected to be faulty, it is detected whether a target camera used by the target software is faulty; and in the case where the target camera used by the target software is not faulty, it is determined that the target software itself is faulty.

[0132] The fault handling device provided in the embodiments of the present application provides users with privacy control by setting a privacy switch. When a vehicle camera fault or a software fault related to the vehicle camera is detected, a policy for externally reporting the fault information is determined based on the privacy switch status. When the privacy switch is on, at least no user private information is transmitted externally, thereby preventing the leakage of user private information and improving the security of user privacy.

[0133] The fault handling device provided in this embodiment is based on the same concept as the fault handling method provided in the above embodiments of this application. It can execute the fault handling method provided in any of the above embodiments of this application and has the corresponding functional modules and beneficial effects. For technical details not fully described in this embodiment, please refer to the specific processing content of the fault handling method provided in the above embodiments of this application and will not be repeated here.

[0134] It should be understood that the modules in the above fault handling device can be implemented in the form of a processor calling software. For example, the device includes a processor, the processor is connected to a memory, and the memory stores instructions. The processor calls the instructions stored in the memory to implement any of the above methods or realize the functions of each unit of the device. The processor can be a general-purpose processor, such as a CPU or a microprocessor, and the memory can be a memory within the device or a memory outside the device. Alternatively, the units in the device can be implemented in the form of hardware circuits. The functions of some or all units can be realized by designing the hardware circuits. The hardware circuit can be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units can be realized by designing the logical relationships between the components within the circuit. For another example, the hardware circuit can be implemented by a PLD. For example, an FPGA can include a large number of logic gate circuits. The connection relationships between the logic gate circuits are configured through a configuration file to realize the functions of some or all of the above units. All units of the above device can be implemented entirely in the form of a processor calling software, or entirely in the form of hardware circuits, or partially in the form of a processor calling software, with the remaining parts implemented in the form of hardware circuits.

[0135] In an embodiment of the present application, a processor is a circuit with the ability to process signals. In one implementation, the processor may be a circuit with the ability to read and execute instructions, such as a CPU, a microprocessor, a GPU, or a DSP. In another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit may be fixed or reconfigurable, such as a hardware circuit implemented by an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the configuration of the hardware circuit can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, DPU, etc.

[0136] It can be seen that each unit in the above device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0137] In addition, the various units in the above apparatus may be fully or partially integrated together, or may be implemented independently. In one implementation, these units are integrated together and implemented in the form of a system-on-chip (SOC). The SOC may include at least one processor for implementing any of the above methods or implementing the functions of the various units of the apparatus. The at least one processor may be of different types, such as a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.

[0138] Exemplary electronic devices

[0139] The present application also provides an electronic device, such as Figure 5 As shown, the electronic device includes: a memory 500 and a processor 510.

[0140] The memory 500 is connected to the processor 510 and is used to store programs.

[0141] The processor 510 is configured to implement the fault handling method in the above embodiment by running the program stored in the memory 500 .

[0142] Specifically, the electronic device may further include: a communication interface 520 , an input device 530 , an output device 540 and a bus 550 .

[0143] The processor 510, the memory 500, the communication interface 520, the input device 530 and the output device 540 are interconnected via a bus.

[0144] Bus 550 may include a pathway for transferring information between the various components of the computer system.

[0145] Processor 510 can be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, or the like, or an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present invention. Alternatively, it can be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware components.

[0146] The processor 510 may include a main processor, and may also include a baseband chip, a modem, and the like.

[0147] The memory 500 stores a program for executing the technical solution of the present invention, and may also store an operating system and other key services. Specifically, the program may include program code, which includes computer operating instructions. More specifically, the memory 500 may include read-only memory (ROM), other types of static storage devices that can store static information and instructions, random access memory (RAM), other types of dynamic storage devices that can store information and instructions, disk storage, flash, etc.

[0148] The input device 530 may include a device for receiving data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor.

[0149] Output device 540 may include devices that allow information to be output to a user, such as a display screen, printer, speakers, etc.

[0150] The communication interface 520 may include any device such as a transceiver to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0151] The processor 510 executes the program stored in the memory 500 and calls other devices, which can be used to implement the various steps of the fault handling method provided in the above embodiments of the present application.

[0152] Example Vehicle

[0153] The present application also provides a vehicle, for example, Figure 6 As shown, the vehicle 600 includes: a memory 601 and a processor 602, wherein the memory 601 stores an executable program code 6011, and the processor 602 is used to call and execute the executable program code 6011 to perform the fault handling method provided in the above embodiment of the present application.

[0154] In the embodiments of the present application, the functional modules of the vehicle can be divided according to the above-mentioned method examples. For example, each functional module can be mapped to a specific functional module, or two or more functions can be integrated into a single processing module. The integrated module can be implemented in the form of hardware. It should be noted that the module division in the embodiments of the present application is illustrative and is only a logical functional division. In actual implementation, other division methods may be used.

[0155] In the case of dividing each functional module into corresponding functional modules, the vehicle may include: a detection module 401 and a processing module 402. It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0156] The vehicle provided in an embodiment of the present application is used to execute the above-mentioned fault handling method, and thus can achieve the same effect as the above-mentioned implementation method.

[0157] In the case of an integrated unit, the vehicle may include a processing module and a storage module. The processing module may be used to control and manage the vehicle's movements, while the storage module may be used to support the vehicle's execution of program codes and data.

[0158] The processing module may be a processor or controller that implements or executes various exemplary logic blocks, modules, and circuits disclosed herein. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processing system (DSP) and a microprocessor, and the storage module may be a memory.

[0159] Exemplary computer program products and storage media

[0160] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps in the fault handling method described in the embodiment of the present application.

[0161] The computer program product may be implemented in hardware, software, or a combination thereof. In one embodiment, the computer program product is implemented as a computer storage medium. In another embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0162] The computer program product may be written in any combination of one or more programming languages ​​to implement the program code for performing the operations of the embodiments of the present application, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0163] In addition, an embodiment of the present application may also be a storage medium on which a computer program is stored, and the computer program is executed by a processor to execute the steps of the fault handling method described in the embodiment of the present application.

[0164] In addition, an embodiment of the present application may also be a chip, which includes a processor and a data interface. The processor reads instructions stored in the memory through the data interface to execute the steps in the fault handling method described in the embodiment of the present application.

[0165] For the sake of simplicity, the aforementioned method embodiments are described as a series of action combinations. However, those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0166] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similarities between the various embodiments can be referred to in conjunction with each other. For device embodiments, since they are generally similar to method embodiments, their description is relatively simple, and for relevant details, reference can be made to the description of the method embodiments.

[0167] The steps in the methods of each embodiment of the present application can be adjusted in sequence, merged, and deleted according to actual needs, and the technical features recorded in each embodiment can be replaced or combined.

[0168] The modules and sub-modules in the devices and terminals in the various embodiments of the present application can be merged, divided, and deleted according to actual needs.

[0169] In the several embodiments provided in this application, it should be understood that the disclosed terminals, devices, and methods can be implemented in other ways. For example, the terminal embodiments described above are merely illustrative. For example, the division of modules or submodules is merely a logical function division. In actual implementation, there may be other division methods, such as multiple submodules or modules can be combined or integrated into another module, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or module, which can be electrical, mechanical or other forms.

[0170] The modules or submodules described as separate components may or may not be physically separate, and the components of the modules or submodules may or may not be physical modules or submodules, that is, they may be located in one place or distributed across multiple network modules or submodules. Some or all of the modules or submodules may be selected to achieve the purpose of this embodiment according to actual needs.

[0171] In addition, each functional module or submodule in each embodiment of the present application may be integrated into a processing module, or each module or submodule may exist physically separately, or two or more modules or submodules may be integrated into a single module. The above-mentioned integrated modules or submodules may be implemented in the form of hardware or software functional modules or submodules.

[0172] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0173] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, software units executed by a processor, or a combination of the two. The software units may be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0174] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

Claims

1. A fault handling method, characterized in that: The method comprises: Detecting the status of a target camera and target software; wherein the target camera refers to a vehicle camera; and the target software refers to software that needs to use data collected by the target camera; When a failure of the target camera and / or the target software is detected, a policy for externally broadcasting the failure information is determined according to the privacy switch status; wherein the privacy switch status is used to indicate whether to transmit user privacy information externally.

2. The fault handling method according to claim 1, characterized in that: Determining a strategy for reporting fault information to the outside world based on the privacy switch status includes: When the privacy switch is in the on state, it is determined that the fault information is not broadcast externally.

3. The fault handling method according to claim 1 or 2, characterized in that: In the case where the target software includes driver monitoring system software, when a fault of the target camera or the target software is detected, determining a strategy for externally broadcasting the fault information according to the privacy switch state includes: When a target camera failure or a target software failure is detected, a strategy for externally broadcasting the fault information is determined based on the privacy switch status and the switch status of the driver monitoring system.

4. The fault handling method according to claim 3, characterized in that: When a target camera failure or a target software failure is detected, a strategy for externally broadcasting the failure information is determined based on the privacy switch status and the switch status of the driver monitoring system, including: When a failure of the target camera or the target software is detected, if the privacy switch is in the on state, determining not to broadcast the failure information externally; In the event of a first camera failure or a driver monitoring system software failure being detected, if the privacy switch is in the off state and the driver monitoring system switch is in the on state, determining to broadcast a first fault message externally; wherein the first camera is a target camera used by the driver monitoring system, and the first fault message is used to instruct the driver monitoring system to stop operating; In the case of detecting a failure of the target camera, if the privacy switch and the switch of the driver monitoring system are both in the off state, determining to broadcast a second failure message externally; wherein the second failure message is used to indicate the failure of the target camera; When a target software failure is detected, if both the privacy switch and the switch of the driver monitoring system are in the off state, it is determined that the failure information will not be broadcast externally.

5. The fault handling method according to claim 4, characterized in that: The second fault information is also used to indicate that the target software using the faulty target camera cannot be used normally.

6. The fault handling method according to claim 3, characterized in that: When a target camera failure or a target software failure is detected, a strategy for reporting the failure information to the outside is determined based on the privacy switch status and the switch status of the driver monitoring system, including: In the case where a fault in the target software is detected, if the switch states of the privacy switch and the driver monitoring system are both in the off state, determining the degree of impact of the target software on vehicle driving; When the impact of the target software on vehicle driving is greater than a preset value, determining to broadcast third fault information externally; wherein the third fault information is used to indicate the target software fault; If the impact of the target software on vehicle driving is less than or equal to a preset value, it is determined not to broadcast the fault information externally.

7. The fault handling method according to claim 1, characterized in that: Determining a strategy for reporting fault information to the outside world based on the privacy switch status includes: When the privacy switch is in the on state, the user privacy information in the fault information is removed, and the fault information with the user privacy information removed is broadcasted externally.

8. The fault handling method according to claim 1, characterized in that: The detecting the status of the target camera and the target software includes: In the case where it is detected that the target software cannot work normally, detecting whether a target camera used by the target software is faulty; When the target camera used by the target software is not faulty, it is determined that the target software itself is faulty.

9. A vehicle, characterized in that: include: memory and processor; The memory is connected to the processor and is used to store programs; The processor is configured to implement the fault handling method according to any one of claims 1 to 8 by running the program in the memory.

10. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the fault handling method according to any one of claims 1 to 8 is implemented.

Images