Vehicle maintenance monitoring method and vehicle

Abstract

The invention provides a vehicle maintenance monitoring method and a vehicle, and is applied to the technical field of vehicle monitoring, and the method comprises the steps: responding to the starting of a maintenance mode of the vehicle, starting at least one camera device, and disabling other camera devices except the started at least one camera device; and collecting monitoring data aiming at the maintenance personnel through the started at least one camera device. Monitoring of a maintenance area is effectively focused, so that the leakage risk of other camera devices to maintenance personnel in the maintenance process is reduced; the collected monitoring data is ensured to be used for maintenance quality control, maintenance investigation and other purposes that the monitoring data must be called, and a user is prevented from being in direct contact with the monitoring data. While the vehicle maintenance quality and safety are ensured, the rights and interests of maintenance personnel and users are protected, and unnecessary legal and moral risks are avoided.

Description

Technical Field

[0001] The present application relates to the field of vehicle monitoring technology, and in particular to a vehicle maintenance monitoring method and a vehicle. Background Art

[0002] As automotive technology continues to advance, in order to ensure that vehicles are properly maintained and repaired, they usually enter maintenance mode, which allows maintenance technicians to perform diagnosis and repairs. However, in maintenance mode, some vehicles do not disable the dashcam function, which will record the conversations or behaviors of maintenance personnel during the maintenance process, causing user complaints or legal issues. Summary of the invention

[0003] In view of this, the purpose of this application is to propose a vehicle maintenance monitoring method and a vehicle to ensure that sensitive information is not leaked, avoid user complaints or legal issues.

[0004] Based on the above purpose, the present application provides a vehicle maintenance monitoring method, comprising:

[0005] In response to the maintenance mode of the vehicle being turned on, enabling at least one camera device and disabling other camera devices except the at least one camera device that is enabled;

[0006] The monitoring data for the maintenance personnel is collected by the at least one enabled camera device.

[0007] Based on the same inventive concept, the present application also provides a vehicle, the vehicle includes an electronic device, the electronic device includes a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the method described above when executing the program.

[0008] From the above description, it can be seen that the present application provides a vehicle maintenance monitoring method and vehicle, wherein the method includes: in response to the maintenance mode of the vehicle being turned on, at least one camera device is enabled and other cameras except the enabled camera device are disabled; monitoring data for maintenance personnel is collected through the enabled camera device; so as to effectively focus on the monitoring of the maintenance area, thereby reducing the risk of leakage of the maintenance process of maintenance personnel by other cameras; ensuring that the collected monitoring data is used for maintenance quality control, maintenance investigation and other purposes that require the use of monitoring data, avoiding users from directly contacting the monitoring data. While ensuring the quality and safety of vehicle maintenance, it also protects the rights and interests of maintenance personnel and users, and avoids unnecessary legal and moral risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0010] Figure 1 A flow chart of a vehicle maintenance monitoring method according to an embodiment of the present application;

[0011] Figure 2 This is a flow chart of a vehicle maintenance monitoring method according to another embodiment of the present application;

[0012] Figure 3 This is a flow chart of a vehicle maintenance monitoring method according to another embodiment of the present application;

[0013] Figure 4 This is a flow chart of a vehicle maintenance monitoring device according to an embodiment of the present application;

[0014] Figure 5 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0015] Figure 6 This is a structural block diagram of the vehicle maintenance monitoring system according to an embodiment of the present application. DETAILED DESCRIPTION

[0016] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0017] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0018] In the related art, with the continuous advancement of automobile technology, vehicle maintenance and repair have become increasingly complex, especially for vehicles equipped with advanced electronic systems and intelligent functions. In order to ensure that the vehicle is properly maintained and repaired, specific technical tools and methods are usually adopted. One of the important functions is the maintenance mode, which allows maintenance technicians to access the deep system settings and functions of the vehicle for diagnosis and repair. In maintenance mode, some vehicle functions may be restricted or disabled to protect the security of the vehicle system and prevent the leakage of sensitive information. However, some vehicles do not disable the dash cam function, which may record the conversation or behavior of the maintenance personnel during the maintenance process, which may contain inappropriate remarks or sensitive information, causing user complaints or legal issues; however, completely disabling the dash cam function also brings some problems. For example, if the maintenance personnel have an accident during the maintenance process or the maintenance personnel perform improper maintenance operations, the lack of video evidence may make it impossible to verify the problem. In addition, if other monitoring equipment in the maintenance area fails or fails to cover all areas, it may also cause monitoring blind spots, thereby affecting the investigation and handling of accidents.

[0019] On the other hand, if the camera is not disabled and the maintenance process is continuously monitored, it can provide necessary evidence for accidents and disputes, but it also poses the risk of leaking sensitive information. Maintenance personnel may discuss technical details about vehicle maintenance, parts costs, and other information that may be considered trade secrets during work. In addition, the maintenance personnel's personal words and deeds may also be inadvertently recorded, which may also lead to legal and ethical issues if they are improperly handled or disseminated.

[0020] Based on the above problems, the applicant discovered that multiple cameras can be set on the vehicle, and the camera devices can be driving recorders. In response to the maintenance mode of the vehicle being turned on, at least one camera device is enabled and other camera devices except the at least one enabled camera device are disabled; monitoring data for maintenance personnel is collected through the at least one enabled camera device; sensitive information is ensured not to be leaked, disputes between users and maintenance personnel are avoided, and data about the maintenance process can be retained.

[0021] It should be noted that if Figure 6 As shown, when the maintenance mode of the vehicle is turned on, the monitoring data of the maintenance personnel is obtained through the enabled camera device and transmitted to the processor, which determines the target monitoring data for the maintenance personnel and determines whether there is sensitive information in the target monitoring data. When there is sensitive information in the target monitoring data, the sensitive information is desensitized and the desensitized target monitoring data is saved to the cloud; wherein, after the enabled camera device obtains the monitoring data for the maintenance personnel, the monitoring data is also saved to the cloud.

[0022] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0023] In some embodiments, Figure 1 As shown, a vehicle maintenance monitoring method is performed by a vehicle controller or a data processor independently provided with the vehicle controller. The subsequent embodiments are all illustrated by taking the vehicle controller as an example. The vehicle includes a plurality of camera devices. The method includes:

[0024] S101, in response to the maintenance mode of the vehicle being turned on, enabling at least one camera device and disabling other camera devices except the at least one enabled camera device;

[0025] In specific implementation, the vehicle's maintenance mode is a special operating mode of the vehicle, designed to protect the vehicle's electronic systems and data security when the vehicle is undergoing maintenance or repair, while ensuring monitoring and control during the repair process. The maintenance mode usually limits or disables certain functions, while enabling necessary maintenance and diagnostic functions. The option to enter the maintenance mode can usually be found through the settings menu of the vehicle information system; for some models that require technical equipment, it may be necessary to use a diagnostic tool to activate the maintenance mode; some vehicles may need to use a specific key or switch combination during the startup process to activate the maintenance mode; in some cases, entering the maintenance mode may need to be performed through an authorized service center to ensure that all operations meet the manufacturer's safety and technical standards. In maintenance mode, certain functions of the vehicle, such as the entertainment system, wireless communication, etc., can be disabled to prevent accidental damage or interference during the maintenance process, and the driving recorder, etc. can also be disabled to help protect the vehicle's sensitive data or prevent unauthorized access or tampering during the maintenance process. In maintenance mode, the vehicle's diagnostic system will provide more detailed information to help technicians accurately diagnose problems and perform effective repairs. In some cases, entering the maintenance mode may be a requirement to comply with laws or safety regulations to ensure that the maintenance process does not negatively affect the safety performance of the vehicle.

[0026] In maintenance mode, enabling specific cameras and disabling other cameras can effectively focus on monitoring the maintenance area, thereby reducing the risk of other cameras leaking the maintenance process to maintenance personnel. In addition, the following aspects can be used to optimize and achieve better results: accurately select the location and number of enabled cameras to ensure that they can cover key maintenance areas. Establish strict data access and processing protocols to ensure that all collected monitoring data is only used for maintenance quality control and accident investigation purposes, and is strictly prohibited from being used for any other purpose. While ensuring the quality and safety of vehicle maintenance, it also protects the rights and interests of maintenance personnel and customers and avoids unnecessary legal and ethical risks.

[0027] S102: Collect monitoring data on maintenance personnel through the at least one enabled camera device.

[0028] In specific implementation, when at least one camera device is enabled, the enabled camera device can be used to collect monitoring data for maintenance personnel. High-quality video recording technology can be used for the enabled camera device to ensure that clear images and audio records can be provided when needed to facilitate accident investigation and evidence collection.

[0029] In this embodiment, if an accident or technical error occurs during the maintenance process, high-quality monitoring data can provide key evidence to help analyze the cause of the accident, so that corresponding improvement measures can be taken to prevent similar incidents from happening in the future. At the same time, enabling at least one camera device and disabling other cameras except the at least one camera device that is enabled can effectively focus on monitoring the maintenance area, thereby reducing the risk of other cameras leaking the maintenance process of maintenance personnel, and avoiding disputes between users and maintenance personnel over the content of the monitoring data after obtaining the monitoring data of the maintenance process of maintenance personnel.

[0030] In some embodiments, the method further comprises:

[0031] In response to the vehicle being in a non-maintenance mode, the at least one enabled camera device is disabled, and the other disabled camera devices are enabled.

[0032] In specific implementation, if the vehicle is in non-maintenance mode, it indicates that the vehicle is not in maintenance state, but in normal use or driving state, and the working state of the camera device is adjusted accordingly, that is, some camera devices specifically used to collect monitoring data may be enabled in maintenance mode. When the vehicle exits maintenance mode and enters non-maintenance mode, the enabled camera devices will be disabled because they are not necessary for normal driving and may also interfere with normal vehicle operation. In non-maintenance mode, different camera devices are needed to improve driving safety and convenience, such as reversing cameras, front cameras, side view cameras, etc. These cameras may be disabled in maintenance mode because they are not necessary for maintenance work. When the vehicle switches back to non-maintenance mode, these camera devices will be re-enabled to restore normal functions and help the driver better monitor the environment around the vehicle.

[0033] In this embodiment, in the non-maintenance mode, it is not necessary to collect monitoring data of the maintenance process of the maintenance personnel, so at least one of the enabled camera devices is disabled and the other disabled camera devices are enabled so that the vehicle can better adapt to different usage scenarios.

[0034] In some embodiments, the method further comprises:

[0035] Obtain monitoring data for maintenance personnel;

[0036] Based on the working status of the maintenance personnel, target monitoring data is determined in the monitoring data, wherein the working status includes a maintenance status and a non-maintenance status.

[0037] In specific implementation, monitoring data can be obtained through cameras installed on the vehicle and enabled to monitor the activities and working environment of maintenance personnel in real time. According to the working status of maintenance personnel, distinguish which data is directly related to maintenance activities (maintenance status) and which data is related to non-work activities (non-maintenance status). After determining the target monitoring data, the target monitoring data can be used to analyze the work efficiency of maintenance personnel and identify the reasons for inefficiency, such as improper use of tools, excessive non-working time, etc.; identify maintenance personnel who need to improve work processes and environments to improve work efficiency based on monitoring data; ensure that maintenance work complies with safety standards, and identify and resolve potential safety issues in a timely manner. When processing monitoring data, especially data involving personal behavior monitoring, privacy protection and legality must be taken into account. Ensure that all monitoring activities comply with local laws and regulations, and that maintenance personnel have a full understanding and consent to the monitoring policy.

[0038] In this embodiment, by monitoring the behavior of maintenance personnel during work, target monitoring data of maintenance personnel in the maintenance state is screened out to provide stable and reliable data support for subsequent analysis of the target monitoring data, thereby avoiding the waste of computing resources caused by analyzing all monitoring data.

[0039] In some embodiments, Figure 2 As shown, the monitoring data includes a video, the target monitoring data includes a target video, the video includes a plurality of frame group images, and each of the frame group images is composed of images of a preset number of frames;

[0040] The determining target monitoring data in the monitoring data based on the working status of the maintenance personnel includes:

[0041] S201, determining the working status of the maintenance personnel in each frame group image;

[0042] In specific implementation, the video is usually composed of continuous frames, each frame is a static image; in this embodiment, the video is divided into a plurality of frame group images, each frame group is composed of a preset number of frames (exemplarily, the preset number of frames can be set to 24 frames). Dividing multiple frame group images helps to process and analyze a large amount of data, because a group of frames can be analyzed as a unit, rather than processing each frame separately, to avoid fluctuations in the working state of the determined maintenance personnel, resulting in the video data obtained by subsequent processing being not coherent enough, affecting the viewing experience. To determine the working state of the maintenance personnel in the frame group image, a computer vision algorithm (such as a convolutional neural network) can be used to analyze the posture and movement of the maintenance personnel in the frame group image. For example, if the maintenance personnel is using tools, bending over, or operating machinery, it is judged to be in a maintenance state. In this way, segments directly related to maintenance activities can be effectively extracted from a large number of surveillance videos, thereby providing data support for the evaluation and optimization of maintenance work. At the same time, this method also helps to improve the efficiency of video data processing.

[0043] S202, in response to determining that the working states of the maintenance personnel in two adjacent frame groups of images are inconsistent, taking a node between the two frame groups of images as a cutting point;

[0044] In specific implementation, in order to process and analyze video data more efficiently, the video is divided into multiple frame group images. Each frame group contains a preset number of consecutive frames. This combination helps to reduce the complexity of processing while retaining enough information to analyze the behavior and working status of maintenance personnel; when the working status displayed by two adjacent frame groups is inconsistent, it indicates that the activities or behaviors of the maintenance personnel have changed significantly, and the node between the two frame groups is marked as a cut point. The cut point is the basis for video segmentation, indicating that the video will be cut into two independent segments at this point. By determining the cut point in response to changes in the working status of maintenance personnel, more fine-grained video analysis can be provided, and the analytical value of the video is also enhanced.

[0045] S203, cutting the video into a plurality of video segments according to the cutting points, each of the video segments having a working status label of the maintenance personnel;

[0046] In specific implementation, based on the determined cutting points, the video will be automatically divided into multiple segments, each cutting point represents a state transition, ensuring that the working state of the maintenance personnel in each video segment is consistent. This ensures the continuity of the video data and the consistency of the state, making subsequent analysis and processing more accurate. At the same time, each video segment will be marked with a corresponding working state label, which may include but is not limited to maintenance state and non-maintenance state.

[0047] S204: In response to determining that the working status tag of the video segment is a maintenance status, determining that the video segment is a target video.

[0048] In specific implementation, when the working status label of the video clip is the maintenance status, the video clip is determined to be the target video to ensure more accurate data analysis, such as determining whether there is sensitive information in the target monitoring data. If there is sensitive information in the target monitoring data, the sensitive information is desensitized and the desensitized target monitoring data is sent to the user end. If there is no sensitive information in the target monitoring data, the target monitoring data is sent to the user end.

[0049] In this embodiment, by dividing the video into segments containing consistent working states, data can be processed and analyzed more efficiently, focusing on video segments directly related to maintenance activities, thereby saving processing time and computing resources, and directly determining whether sensitive information exists in the target monitoring data.

[0050] In some embodiments, the monitoring data may include only images, the target monitoring data includes target images, and determining the target monitoring data in the monitoring data based on the working status of the maintenance personnel includes:

[0051] The working state of the maintenance personnel in the image is determined, and in response to the maintenance personnel in the image being in a maintenance state, the image is determined to be a target image.

[0052] In specific implementation, a computer vision algorithm (such as a convolutional neural network) can be used to analyze the posture and movements of the maintenance personnel in the image. For example, if the maintenance personnel is using tools, bending over, or operating machinery, it is determined that the maintenance personnel in the image is in a maintenance state.

[0053] In some embodiments, determining the working status of the maintenance personnel in each frame group image includes:

[0054] Determine the working status of the maintenance personnel in each frame of a frame group, and determine the ratio of the number of frames of the image in which the maintenance personnel is in the maintenance state to the preset number of frames;

[0055] In the specific implementation, firstly, continuous video frames are obtained from a group of images. Before processing, these video frames may need to be preprocessed, including adjusting the resolution, cutting unnecessary edges, adjusting the lighting conditions, etc., to improve the accuracy and efficiency of subsequent analysis. Then, face recognition, human posture estimation or other related computer vision technologies are used to identify and track maintenance personnel in each frame, including segmenting the outline of personnel from the video frame, identifying specific work clothes or tools, and tracking the movement trajectory of personnel in the video. For each identified maintenance personnel in each frame, further analyze their posture, hand position, tools used and other information to determine whether they are in a maintenance state. This can be achieved by setting some predefined rules. For example, if the maintenance personnel holds tools and the body is tilted to a certain angle, it may indicate that they are carrying out maintenance activities. In a group of images, the number of frames in the maintenance state is counted, and the ratio is calculated by the number of frames in the maintenance state and the preset number of frames. For example, the preset number of frames is 100, of which 70 frames show that the maintenance personnel are in the maintenance state, and the ratio is 0.7.

[0056] In response to the ratio being greater than or equal to a preset ratio, it is determined that the maintenance personnel in the one-frame group of images is in a maintenance state.

[0057] In a specific implementation, when the ratio of the number of frames in the maintenance state to the preset number of frames reaches or exceeds the preset ratio (exemplarily, the preset ratio can be set to 0.6), it is considered that the maintenance personnel are mainly in the maintenance state in a frame group image.

[0058] In this embodiment, the working status of the maintenance personnel in a frame group of images is determined by the ratio of the frame number of images of the maintenance personnel in the maintenance state to the preset frame number, thereby ensuring the consistency of the working status of the maintenance personnel in the frame group of images and providing a reliable data basis for subsequent analysis.

[0059] In some embodiments, determining the working status of the maintenance personnel in each frame of the group of images includes:

[0060] Perform posture recognition on the maintenance personnel in a frame of image to obtain the probability that the posture of the maintenance personnel is a maintenance posture;

[0061] In specific implementation, a frame of image can be preprocessed, including denoising, contrast enhancement, etc., to improve image quality and ensure the accuracy of subsequent processing. Then, an image recognition algorithm (such as ResNet50) is applied to identify the posture of the maintenance personnel and obtain the probability that the posture of the maintenance personnel is a maintenance posture.

[0062] In response to determining that the probability is greater than or equal to a preset probability, determining that the maintenance personnel in the one frame of image is in a maintenance state;

[0063] In specific implementation, the probability that the maintenance personnel's posture is a maintenance posture indicates the degree of matching between the current posture and the maintenance posture. If the probability is greater than or equal to a preset probability (exemplarily, the preset probability can be set to 0.6), it is confirmed that the maintenance personnel is currently in a maintenance state.

[0064] In response to determining that the probability is less than a preset probability, the working state of the maintenance personnel in the one frame of image is determined based on the positional relationship between the maintenance personnel and the vehicle component.

[0065] In specific implementation, the relative position of the maintenance personnel and the vehicle parts can be captured by various sensors (such as cameras, infrared sensors, etc.) to analyze whether the maintenance personnel are in a specific maintenance area or whether they are in an appropriate proximity to a specific vehicle part. If the maintenance personnel are close to a key vehicle part, especially a part that usually requires manual operation, it can be determined that the maintenance personnel may be performing maintenance work. By using the position relationship to determine the working status of the maintenance personnel, the working status of the maintenance personnel can be monitored more comprehensively and accurately through the comprehensive use of posture recognition and position analysis.

[0066] In this embodiment, posture recognition alone may not accurately determine the maintenance status in some cases, especially when the posture is not obvious or blocked. Combined with position relationship analysis, when posture recognition is not sufficient to provide accurate information, the relative position of the maintenance personnel and key vehicle components can be analyzed to supplement or confirm their working status, thereby improving the accuracy of identifying the working status of the maintenance personnel in a frame of image.

[0067] In some embodiments, determining the working status of the maintenance personnel in the one frame of image based on the positional relationship between the maintenance personnel and the vehicle component includes:

[0068] Performing target detection on the maintenance personnel and the vehicle component in the one frame of image, obtaining the boundary boxes of the maintenance personnel and the vehicle component respectively, and determining the overlap rate between the boundary box of the maintenance personnel and the boundary box of the vehicle component;

[0069] In specific implementation, the target detection algorithm is used to identify the maintenance personnel and vehicle parts in the image; through the pre-trained human detection model (such as YOLO, SSD, Faster R-CNN, etc.), the human body in the image can be identified and a bounding box can be generated for it, that is, the position of the maintenance personnel is framed in the image. Similarly, the detection model trained for vehicle parts is used to identify and locate the vehicle parts in the image, such as the engine, tires, etc., and generate a bounding box for them. When the bounding boxes of the maintenance personnel and the vehicle parts are obtained, the overlap rate of the two bounding boxes is calculated, which is usually obtained by calculating the percentage of the intersection of the two bounding boxes to the union.

[0070] In response to determining that the overlap rate is greater than or equal to a preset overlap rate, determining that the maintenance personnel in the one frame of image is in a maintenance state;

[0071] In response to determining that the overlap rate is less than a preset overlap rate, it is determined that the maintenance personnel in the one frame of image is in a non-maintenance state.

[0072] In specific implementation, the overlap rate is an intuitive indicator to measure the spatial proximity of two bounding boxes. In this scenario, the bounding boxes represent maintenance personnel and vehicle parts. A high overlap rate indicates that the body parts of the maintenance personnel are very close to the vehicle parts in space, usually because the maintenance personnel need to operate or inspect these parts. Maintenance work usually involves direct physical contact with vehicle parts, such as replacing parts, making adjustments, checking functions, etc. When the overlap rate of the bounding boxes of the maintenance personnel and the vehicle parts is greater than or equal to the preset overlap rate (exemplarily, the preset overlap rate is set to 10%), it can be determined that the maintenance personnel are performing maintenance, that is, the maintenance personnel are in a maintenance state in a frame of image; when the overlap rate of the bounding boxes of the maintenance personnel and the vehicle parts is less than the preset overlap rate (exemplarily, the preset overlap rate is set to 10%), it can be determined that the maintenance personnel are not performing maintenance, that is, the maintenance personnel are in a non-maintenance state in a frame of image.

[0073] In this embodiment, the spatial relationship between the maintenance personnel and the vehicle components is precisely calculated to more accurately determine whether the maintenance personnel are performing specific maintenance activities; the use of the bounding box overlap rate as a quantitative indicator can reduce misjudgment and provide a more objective evaluation standard.

[0074] In some embodiments, after determining the target monitoring data, the method further includes:

[0075] Determining whether there is sensitive information in the target monitoring data;

[0076] In specific implementation, the definition of sensitive information includes but is not limited to personal identity information, private data of vehicle owners, trade secrets (such as maintenance techniques, parts costs, etc.), or any information that may violate privacy protection regulations. The process of analyzing and identifying sensitive information can be implemented by relying on advanced data processing technologies, such as machine learning algorithms to identify and classify sensitive information, to ensure that sufficiently transparent monitoring data and supervision of the maintenance process are preserved without leaking sensitive information.

[0077] In response to the existence of sensitive information in the target monitoring data, the sensitive information is desensitized and the desensitized target monitoring data is saved and allowed to be viewed by a user.

[0078] In practice, desensitization refers to the processing of sensitive information to remove or mask its sensitive features while retaining the data's usefulness. This process is critical to complying with privacy laws and regulations, especially where personal data and trade secrets are involved. In the context of vehicle maintenance, desensitization ensures transparency and monitoring of the maintenance process while protecting the personal privacy of the owner and maintenance personnel and the sensitive information of the vehicle. Common methods of desensitization include: partially replacing sensitive data with falsified, unrecognizable information, such as replacing the owner's name or license plate number with random characters or asterisks; or deleting sensitive information, especially when the information does not affect the overall purpose of the data; or converting data from specific values ​​to broader categories to reduce the precision of the data, such as generalizing a specific range of maintenance costs to a larger range of costs. The target monitoring data after desensitization is saved to the cloud and allowed to be viewed by users. Normally, the target monitoring data is not actively sent to users. However, if users need to view the maintenance process in some special circumstances, they can view it through the target monitoring data without access to any sensitive data. This not only protects the privacy of maintenance personnel, but also helps to enhance users' trust and satisfaction with the maintenance process, providing users with a transparent and reliable service experience.

[0079] In some embodiments, Figure 3 As shown, the determining whether there is sensitive information in the target monitoring data includes:

[0080] S301, acquiring the target video, where the target video includes image data and audio data;

[0081] In specific implementation, the target video usually includes image data and audio data. Image data includes all visual information in the video frame, such as people, objects, environment, etc.; image data can be further processed by video analysis technology to detect and identify specific image features or behavior patterns. Audio data includes all sound information in the video, such as dialogue, background noise, specific sound events, etc.

[0082] S302, transcribing the audio data and the image data into text and performing sensitive word detection on the text;

[0083] In specific implementation, audio data to text usually uses deep learning and machine learning algorithms to identify and transcribe the language in the audio. The process includes: removing noise, enhancing speech signals, extracting useful features from audio signals, such as frequency and amplitude, using trained models (such as hidden Markov models, deep neural networks) to convert the extracted features into text, and then correcting grammatical errors to improve the readability and accuracy of the text. The technologies involved in image data to text are mainly computer vision and image recognition, including: identifying objects and entities in images (such as faces, vehicles, text, etc.), and then using optical character recognition (OCR) technology to recognize text information in the image. When audio and image data are transcribed into text, sensitive word detection is performed. First, the transcribed text is subjected to linguistic analysis, including word segmentation, part-of-speech tagging, etc.; the analyzed text is matched with a predefined sensitive word library. Sensitive words may include personal identity information, confidential information, inappropriate or sensitive speech, etc.; in some cases, the contextual use of words needs to be considered to reduce false positives and false negatives; respond after sensitive words are found and mark the content containing sensitive information.

[0084] S303: In response to the presence of sensitive words in the text, determine that sensitive information exists in the target video.

[0085] In specific implementation, when sensitive words exist in the text, it is determined that sensitive information exists in the target video. The sensitive information can be desensitized and the desensitized target monitoring data can be sent to the user end, which not only protects the privacy of maintenance personnel, but also helps to enhance users' trust and satisfaction with the maintenance process, providing users with a transparent and reliable service experience.

[0086] In this embodiment, by detecting and desensitizing sensitive information in video and audio data, such as personal identity information, confidential business information, etc., the privacy of individuals is effectively protected, which helps prevent data leakage and abuse and reduce legal and moral risks. By allowing users to access desensitized monitoring data, the transparency of the maintenance process can be shown to users, allowing users to see the actual situation of the maintenance work, which helps to improve the credibility of the service and user satisfaction.

[0087] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0088] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0089] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a vehicle maintenance monitoring method device.

[0090] refer to Figure 4 , the vehicle maintenance monitoring method device comprises:

[0091] A determination module 701 is configured to enable at least one camera device and disable other camera devices except the enabled at least one camera device in response to the maintenance mode of the vehicle being turned on;

[0092] The processing module 702 is configured to collect monitoring data of the maintenance personnel through the at least one enabled camera device.

[0093] Furthermore, the processing module 702 is further specifically configured to:

[0094] Obtain monitoring data for maintenance personnel;

[0095] Based on the working status of the maintenance personnel, target monitoring data is determined in the monitoring data, wherein the working status includes a maintenance status and a non-maintenance status.

[0096] Furthermore, the processing module 702 is further specifically configured to:

[0097] Determine the working status of the maintenance personnel in each frame group image;

[0098] In response to determining that the working states of the maintenance personnel in two adjacent frame groups of images are inconsistent, taking a node between the two frame groups of images as a cutting point;

[0099] Cutting the video into a plurality of video clips according to the cutting points, each of the video clips having a working status label of the maintenance personnel;

[0100] In response to determining that the working status tag of the video segment is a maintenance status, the video segment is determined to be a target video.

[0101] Furthermore, the processing module 702 is further specifically configured to:

[0102] Determine the working status of the maintenance personnel in each frame of a frame group, and determine the ratio of the number of frames of the image in which the maintenance personnel is in the maintenance state to the preset number of frames;

[0103] In response to the ratio being greater than or equal to a preset ratio, it is determined that the maintenance personnel in the one-frame group of images is in a maintenance state.

[0104] Furthermore, the processing module 702 is further specifically configured to:

[0105] Perform posture recognition on the maintenance personnel in a frame of image to obtain the probability that the posture of the maintenance personnel is a maintenance posture;

[0106] In response to determining that the probability is greater than or equal to a preset probability, determining that the maintenance personnel in the one frame of image is in a maintenance state;

[0107] In response to determining that the probability is less than a preset probability, the working state of the maintenance personnel in the one frame of image is determined based on the positional relationship between the maintenance personnel and the vehicle component.

[0108] Furthermore, the processing module 702 is further specifically configured to:

[0109] Performing target detection on the maintenance personnel and the vehicle component in the one frame of image, obtaining the boundary boxes of the maintenance personnel and the vehicle component respectively, and determining the overlap rate between the boundary box of the maintenance personnel and the boundary box of the vehicle component;

[0110] In response to determining that the overlap rate is greater than or equal to a preset overlap rate, determining that the maintenance personnel in the one frame of image is in a maintenance state;

[0111] In response to determining that the overlap rate is less than a preset overlap rate, it is determined that the maintenance personnel in the one frame of image is in a non-maintenance state.

[0112] Furthermore, the processing module 702 is further specifically configured to:

[0113] Determining whether there is sensitive information in the target monitoring data;

[0114] In response to the existence of sensitive information in the target monitoring data, the sensitive information is desensitized and the desensitized target monitoring data is saved and allowed to be viewed by a user.

[0115] Furthermore, the processing module 702 is further specifically configured to:

[0116] Acquire the target video, where the target video includes image data and audio data;

[0117] Transcribing the audio data and the image data into text and performing sensitive word detection on the text;

[0118] In response to the presence of sensitive words in the text, it is determined that sensitive information exists in the target video.

[0119] For the convenience of description, the above device is described in terms of functions divided into various modules. Of course, when implementing the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0120] The device of the above embodiment is used to implement the corresponding vehicle maintenance monitoring method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0121] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the vehicle maintenance monitoring method described in any of the above embodiments is implemented.

[0122] Figure 5 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0123] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0124] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0125] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0126] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0127] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0128] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0129] The electronic device of the above embodiment is used to implement the corresponding vehicle maintenance monitoring method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0130] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the vehicle maintenance monitoring method described in any of the above embodiments.

[0131] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0132] The computer instructions stored in the storage medium of the above embodiments are used to enable the computer to execute the vehicle maintenance monitoring method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0133] Based on the same concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a computer program product, including computer program instructions. When the computer program instructions are run on a computer, the computer executes the method described in any of the above embodiments, which has the beneficial effects of the corresponding method embodiments and will not be repeated here.

[0134] It is understandable that before using the technical solutions of each embodiment of the present disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved will be informed to the user in an appropriate manner, and the user's authorization will be obtained.

[0135] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly remind the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can independently choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0136] As an optional but non-limiting implementation, in response to receiving the user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0137] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0138] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0139] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0140] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0141] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present application.

Claims

1. A vehicle maintenance monitoring method, characterized in that: The vehicle includes a plurality of camera devices, and the method includes: In response to the maintenance mode of the vehicle being turned on, enabling at least one camera device and disabling other camera devices except the at least one camera device that is enabled; The monitoring data for the maintenance personnel is collected by the at least one enabled camera device.

2. The vehicle maintenance monitoring method according to claim 1, characterized in that: The method further comprises: In response to the vehicle being in a non-maintenance mode, the at least one enabled camera device is disabled, and the other disabled camera devices are enabled.

3. The vehicle maintenance monitoring method according to claim 1, characterized in that: The method further comprises: Obtain monitoring data for maintenance personnel; Based on the working status of the maintenance personnel, target monitoring data is determined in the monitoring data, wherein the working status includes a maintenance status and a non-maintenance status.

4. The vehicle maintenance monitoring method according to claim 3, characterized in that: The monitoring data includes a video, the target monitoring data includes a target video, the video includes a plurality of frame group images, and each of the frame group images is composed of images of a preset number of frames; The determining target monitoring data in the monitoring data based on the working status of the maintenance personnel includes: Determine the working status of the maintenance personnel in each frame group image; In response to determining that the working states of the maintenance personnel in two adjacent frame groups of images are inconsistent, taking a node between the two frame groups of images as a cutting point; Cutting the video into a plurality of video clips according to the cutting points, each of the video clips having a working status label of the maintenance personnel; In response to determining that the working status tag of the video segment is a maintenance status, the video segment is determined to be a target video.

5. The vehicle maintenance monitoring method according to claim 4, characterized in that: Determining the working status of the maintenance personnel in each frame group image includes: Determine the working status of the maintenance personnel in each frame of a frame group, and determine the ratio of the number of frames of the image in which the maintenance personnel is in the maintenance state to the preset number of frames; In response to the ratio being greater than or equal to a preset ratio, it is determined that the maintenance personnel in the one-frame group of images is in a maintenance state.

6. The vehicle maintenance monitoring method according to claim 5, characterized in that: Determining the working status of the maintenance personnel in each frame of a group of images includes: Perform posture recognition on the maintenance personnel in a frame of image to obtain the probability that the posture of the maintenance personnel is a maintenance posture; In response to determining that the probability is greater than or equal to a preset probability, determining that the maintenance personnel in the one frame of image is in a maintenance state; In response to determining that the probability is less than a preset probability, the working state of the maintenance personnel in the one frame of image is determined based on the positional relationship between the maintenance personnel and the vehicle component.

7. The vehicle maintenance monitoring method according to claim 6, characterized in that: The determining the working state of the maintenance personnel in the one frame of image according to the positional relationship between the maintenance personnel and the vehicle component includes: Performing target detection on the maintenance personnel and the vehicle component in the one frame of image, obtaining the boundary boxes of the maintenance personnel and the vehicle component respectively, and determining the overlap rate between the boundary box of the maintenance personnel and the boundary box of the vehicle component; In response to determining that the overlap rate is greater than or equal to a preset overlap rate, determining that the maintenance personnel in the one frame of image is in a maintenance state; In response to determining that the overlap rate is less than a preset overlap rate, it is determined that the maintenance personnel in the one frame of image is in a non-maintenance state.

8. The vehicle maintenance monitoring method according to claim 4, characterized in that: After determining the target monitoring data, it also includes: Determining whether there is sensitive information in the target monitoring data; In response to the existence of sensitive information in the target monitoring data, the sensitive information is desensitized and the desensitized target monitoring data is saved and allowed to be viewed by a user.

9. The vehicle maintenance monitoring method according to claim 8, characterized in that: The determining whether there is sensitive information in the target monitoring data includes: Acquire the target video, where the target video includes image data and audio data; Transcribing the audio data and the image data into text and performing sensitive word detection on the text; In response to the presence of sensitive words in the text, it is determined that sensitive information exists in the target video.

10. A vehicle, characterized in that: The vehicle includes an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 9 when executing the program.

Images