AI camera state self-checking and repairing method and system for excavator
Through real-time monitoring and fault type identification, combined with re-power and wiper processing methods, the independent repair of AI cameras is achieved, solving the problems of insufficient autonomous repair capabilities and poor universality in the existing technology, and improving the efficiency and accuracy of monitoring and repair.
Patent Information
- Application Number
- CN202411913892.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-24
- Publication Date
- 2025-03-25
AI Technical Summary
The existing AI camera status monitoring methods cannot achieve independent repair, and the existing repair methods are poorly universal, making it difficult to prevent secondary damage to the hardware.
By monitoring the working status of the AI camera in real time, determining the fault type, and taking different handling methods according to different fault types, including powering up and starting the wiper, and setting a number of times limits to prevent secondary damage to the hardware.
The independent repair of AI cameras is realized, which reduces human participation, improves monitoring accuracy and repair efficiency, and avoids secondary hardware damage.
Smart Images
Figure CN119697361A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of camera status self-checking, and in particular, relates to an AI camera status self-checking and repairing method and system for an excavator. Background Art
[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.
[0003] Installing AI cameras on production machinery such as excavators can monitor the working status and surrounding environment of production machinery in real time, providing operators and machinery managers with more intuitive and accurate operation information. However, in actual applications, AI cameras may experience equipment failure or abnormal images due to various reasons (such as dust, water mist, collision, etc.), which will not only affect the monitoring effect, but also may cause safety hazards to machinery production.
[0004] To solve this problem, some AI camera status monitoring methods for excavators have emerged. However, these methods still have some technical problems, such as:
[0005] (1) The existing AI camera status monitoring method can only be used to monitor whether the camera's operating status is abnormal. When an abnormal status is detected, an alarm will be issued, and then the technicians will repair the status. This method requires human participation and does not have the function of autonomous repair by the AI camera; therefore, it is time-consuming and labor-intensive and will delay normal production work.
[0006] (2) In order to reduce human intervention as much as possible, some existing AI camera status monitoring methods for excavators choose to re-power on cameras that have already experienced abnormal status. As long as the problem is solved, no human intervention is required. However, there are many reasons for the abnormal working status of the camera, and simply re-powering on the camera cannot solve problems such as stains on the camera; therefore, the existing AI camera status monitoring methods have poor universality and it is difficult to prevent secondary damage to the hardware caused during the repair process. Summary of the invention
[0007] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides an AI camera status self-inspection and repair method and system for excavators, which can determine the type of fault by real-time monitoring the working status of the AI camera, and adopt different processing measures according to different fault types; moreover, a number limit is set for each processing measure to prevent secondary damage to the hardware during the repair process.
[0008] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:
[0009] A first aspect of the present invention provides a method for self-checking and repairing the status of an AI camera for an excavator.
[0010] An AI camera status self-checking and repairing method for an excavator, comprising:
[0011] Collect the operation data of AI cameras;
[0012] The operation state of the AI camera is judged based on the obtained operation data, the sending form of the message is determined according to the obtained operation state, and the fault type of the AI camera is determined according to the sending form of the obtained message; wherein the fault type of the AI camera includes a first fault, a second fault and a third fault;
[0013] According to the fault type of the AI camera, self-inspection and repair are performed. Specifically: when the fault type is the first fault or the third fault, the AI camera is powered on again and the power-on times are counted. If the power-on times are greater than the power-on threshold, the fault information of the fault type is uploaded to the vehicle instrument of the excavator; when the fault type is the second fault, the wiper is started and the single start time of the wiper is set, and the start times of the wiper are counted. If the start times are greater than the start threshold, the fault information of the fault type is uploaded to the vehicle instrument of the excavator.
[0014] Furthermore, the first fault indicates that the working state of the AI camera is abnormal, the second fault indicates that the AI camera is dirty or blocked, and the third fault indicates that the picture of the AI camera is abnormal.
[0015] Furthermore, the fault type of the AI camera is determined based on the sending form of the obtained message. Specifically, the message includes three types of messages, namely, message No. 1, message No. 2 and message No. 3, and a message sending threshold is set before the message is sent.
[0016] Furthermore, when the time for not receiving message No. 1 exceeds the set message sending threshold, it is determined that the AI camera has a first fault; when the received message is in the form of message No. 2, it is determined that the AI camera has a second fault; and when the received message is in the form of message No. 3, it is determined that the AI camera has a third fault.
[0017] Furthermore, among the three fault types, the priority of the first fault is higher than that of the second fault and the third fault, that is: first determine whether the fault type of the AI camera is the first fault, and when it is determined that it does not belong to the first fault, then determine the second fault and the third fault.
[0018] A second aspect of the present invention provides an AI camera status self-inspection and repair system for an excavator.
[0019] An AI camera status self-checking and repairing system for an excavator comprises: an AI camera, a vehicle controller, a body controller, a wiper and a vehicle instrument.
[0020] The AI camera includes a camera body and an MCU processor;
[0021] The MCU processor is configured to: collect operating data of the AI camera, determine the operating status of the AI camera based on the obtained operating data, determine the sending form of the message according to the obtained operating status, and send the message to the vehicle controller;
[0022] The vehicle controller is configured to: determine the fault type of the AI camera in the form of the obtained message, and send the fault information to the vehicle body controller;
[0023] The vehicle body controller is configured to: perform self-inspection and repair according to the fault type of the AI camera and determine whether to upload the fault information to the vehicle instrument according to the result of the self-inspection and repair.
[0024] Furthermore, the AI camera is connected to the vehicle controller via a CAN bus.
[0025] Furthermore, judging whether to upload the fault information to the vehicle instrument according to the result of the self-check and repair includes: setting a repair upper limit during the self-check and repair, and uploading the fault information to the vehicle instrument when and only when the number of self-check and repair exceeds the set repair upper limit.
[0026] Furthermore, different fault types correspond to different repair upper limits. Specifically: when the fault type of the AI camera is the first fault or the third fault, the corresponding repair upper limit is the upper limit of the power-on times; when the fault type of the AI camera is the second fault, the corresponding repair upper limit is the upper limit of the wiper start times.
[0027] One or more of the above technical solutions have the following beneficial effects:
[0028] (1) The present invention collects the operating data of the AI camera, determines the operating status of the AI camera based on the obtained operating data; determines the sending form of the message according to the obtained operating status, and determines the fault type of the AI camera according to the sending form of the obtained message; and performs self-check and repair according to the fault type of the AI camera. Therefore, when the present invention detects that the AI camera has an abnormal working state, the vehicle controller on the excavator can realize self-check and repair of the AI camera without human intervention, saving time and effort, and will not delay normal production work.
[0029] (2) The present invention provides a judgment method for different fault types of AI cameras, and different means can be used to handle different fault types; at the same time, a number limit is set for each processing method to prevent secondary damage to the hardware during the repair process.
[0030] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.
[0032] Figure 1 The present invention is a flowchart of an AI camera status self-check and repair method for an excavator in Embodiment 1 of the present invention.
[0033] Figure 2 The figure is a schematic diagram of the data interaction relationship between the devices included in the AI camera status self-check and repair system in the second embodiment of the present invention. DETAILED DESCRIPTION
[0034] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.
[0035] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.
[0036] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.
[0037] Embodiment 1
[0038] This embodiment discloses a self-checking and repairing method for an AI camera status for an excavator.
[0039] like Figure 1 As shown, a self-checking and repairing method for an AI camera state for an excavator includes:
[0040] Step S1, collecting the operation data of the AI camera;
[0041] Step S2: judging the operating state of the AI camera based on the obtained operating data, determining the sending form of the message according to the obtained operating state, and determining the fault type of the AI camera according to the sending form of the obtained message; wherein the fault types of the AI camera include a first fault, a second fault, and a third fault;
[0042] Step S3, according to the fault type of the AI camera, perform self-inspection and repair, specifically: when the fault type is the first fault or the third fault, re-power on the AI camera and count the number of power-on times. If the number of power-on times is greater than the power-on threshold, upload the fault information of the fault type to the vehicle instrument of the excavator; when the fault type is the second fault, start the wiper and set the single start time of the wiper, count the number of start-up times of the wiper, and if the number of start-up times is greater than the start threshold, upload the fault information of the fault type to the vehicle instrument of the excavator.
[0043] Based on the above process, the present invention can determine the fault type by real-time monitoring of the working status of the AI camera, and adopt different processing methods according to different fault types; moreover, a number limit is set for each processing method to prevent secondary damage to the hardware during the repair process. To facilitate the understanding of the technical solution of the present invention, the specific implementation steps of the technical solution of the present invention are further explained and illustrated below.
[0044] Step S1, collecting the operating data of the AI camera.
[0045] The AI camera consists of a camera body and an MCU processor. The camera body is responsible for capturing the optical images of the excavator's working scene and projecting them onto the internal image sensor. The image sensor can convert the received optical images into electrical signals, which are then converted into digital signals by an analog-to-digital converter (ADC) for subsequent digital processing. The converted digital signals are transmitted to the MCU processor through the camera's output interface (or directly connected to the CAN bus). The MCU processor (microcontroller unit) is used to analyze the received digital signals and exchange data with the vehicle controller on the excavator.
[0046] Step S2: determine the operating status of the AI camera based on the obtained operating data, determine the sending form of the message according to the obtained operating status, and determine the fault type of the AI camera based on the sending form of the obtained message; wherein the fault types of the AI camera include a first fault, a second fault and a third fault.
[0047] Step S2-1: The MCU processor determines the operating status of the AI camera based on the obtained operating data.
[0048] The MCU processor is embedded with a trained artificial neural network (ANN) algorithm for judging the operating status of the AI camera. Specifically, the MCU processor can identify and classify the images collected by the camera body in the AI camera by calling the artificial neural network algorithm, that is, distinguish whether the images collected by the camera body have the characteristic phenomenon unique to the second fault (such as dirt) and whether there are the characteristic phenomena unique to the third fault (such as screen problems). In addition, it should be noted that the artificial neural network algorithm embedded in the MCU processor in this embodiment is an existing learning algorithm, so no further elaboration is made on this.
[0049] Step S2-2, determining the sending form of the message according to the obtained operating status, and determining the fault type of the AI camera according to the sending form of the obtained message.
[0050] The messages are divided into three categories: message 1, message 2 and message 3, and the message sending threshold is set before the message is sent. The MCU processor and the vehicle controller communicate through the CAN bus, that is, the MCU processor sends messages to the vehicle controller through the CAN bus. Specifically:
[0051] The MCU processor continuously sends message No. 1 to the vehicle controller, and at the same time, analyzes and judges the images collected by the camera body based on the trained artificial neural network algorithm. During operation, when the vehicle controller does not receive message No. 1 sent by the MCU processor for a period of time that exceeds the set message sending threshold, it is determined that the AI camera has the first fault; when the MCU processor's judgment result on the image is a unique representation of the second fault, it sends message No. 2 according to the specified communication matrix, and when the message received by the vehicle controller is in the form of message No. 2, it is determined that the AI camera has the second fault; when the MCU processor's judgment result on the image is a unique representation of the third fault, it sends message No. 3 according to the specified communication matrix, and when the message received by the vehicle controller is in the form of message No. 3, it is determined that the AI camera has the third fault.
[0052] In general, the sending cycle of the No. 1 message sent by the MCU processor to the vehicle controller is 1s. To ensure the accuracy of the first fault judgment, the message sending threshold set in this embodiment is 30s, that is, when the vehicle controller does not receive the No. 1 message sent by the MCU processor for more than 30s, it is determined that the AI camera has a first fault. At the same time, this embodiment also specifies the message sending format. Specifically, the No. 1 message sent by the MCU processor to the vehicle controller is formally expressed as 0x18FF0365; the No. 2 message sent by the MCU processor to the vehicle controller is formally expressed as 0x18FF0465; the No. 3 message sent by the MCU processor to the vehicle controller is formally expressed as 0x18FF0565.
[0053] Furthermore, the first fault indicates that the working state of the AI camera is abnormal, the second fault indicates that the AI camera is dirty or blocked, and the third fault indicates that the picture of the AI camera is abnormal; wherein the picture abnormality includes but is not limited to the phenomenon that the camera screen appears black, distorted, and blue. Among the above three types of faults, the priority of the first fault is higher than the second and third faults, that is: first determine whether the fault type of the AI camera is the first fault, and when it is determined that it is not the first fault, then determine the second and third faults.
[0054] Step S3, according to the fault type of the AI camera, perform self-inspection and repair, specifically: when the fault type is the first fault or the third fault, re-power on the AI camera and count the number of power-on times. If the number of power-on times is greater than the power-on threshold, upload the fault information of the fault type to the vehicle instrument of the excavator; when the fault type is the second fault, start the wiper and set the single start time of the wiper, count the number of start-up times of the wiper, and if the number of start-up times is greater than the start threshold, upload the fault information of the fault type to the vehicle instrument of the excavator.
[0055] In this embodiment, when the vehicle controller analyzes the received message data and determines that the fault type of the AI camera is the first fault or the third fault, it will send fault information in the form of a CAN communication message to the body controller. Specifically: when the body controller receives the fault signal of the first fault, the body controller will control the AI camera to power on again. After completing one power-on, the body controller will feedback the response result to the vehicle controller. Each time the vehicle controller receives fault feedback belonging to the first fault, the response feedback of the first fault is increased by 1 and the first fault is re-diagnosed; if the first fault disappears, the vehicle controller will clear the response feedback of the first fault. If the power-on times of the AI camera are greater than the power-on threshold, the vehicle controller will only send the first fault information to the vehicle instrument of the excavator; so that subsequent operators can view it. When the body controller receives the fault signal of the third fault, the body controller will control the AI camera to power on again. After completing one power-on, the body controller will feed back the response result to the vehicle controller. Each time the vehicle controller receives fault feedback belonging to the third fault, the response feedback of the third fault will be increased by 1 and the third fault will be re-diagnosed. If the third fault disappears, the vehicle controller will clear the response feedback of the third fault. If the power-on times of the AI camera are greater than the power-on threshold, the vehicle controller will only send the third fault information to the vehicle instrument of the excavator so that subsequent operators can check it.
[0056] When the vehicle controller analyzes the received message data and determines that the fault type of the AI camera is the second fault, it will send fault information to the body controller. Specifically: when the body controller receives the fault signal of the second fault, the body controller will control the wiper to start, and the start time of each wiper shall not exceed the specified single start time. After completing a wiper start, the body controller will feedback the response result to the vehicle controller. Each time the vehicle controller receives a response feedback of the second fault, the response feedback of the second fault is increased by 1 and the second fault is re-judged; if the second fault disappears, the vehicle controller will clear the response feedback of the second fault; if the number of response feedbacks of the second fault is greater than the set start threshold, the vehicle controller will only send the third fault information to the vehicle instrument of the excavator for subsequent operators to view.
[0057] Further, in this embodiment, the power-on threshold and the start threshold are both set to 3, and the single start time is 3s. It can be understood that the power-on threshold, the start threshold and the start time can be designed according to actual needs, and this embodiment does not specifically limit this.
[0058] Embodiment 2
[0059] This embodiment discloses an AI camera status self-checking and repairing system for an excavator.
[0060] like Figure 2 As shown, an AI camera status self-inspection and repair system for an excavator includes: an AI camera, a vehicle controller, a body controller, wipers and vehicle instruments.
[0061] The AI camera includes a camera body and an MCU processor;
[0062] The MCU processor is configured to: collect operating data of the AI camera, determine the operating status of the AI camera based on the obtained operating data, determine the sending form of the message according to the obtained operating status, and send the message to the vehicle controller;
[0063] The vehicle controller is configured to: determine the fault type of the AI camera in the form of the message sent, and send the fault information to the vehicle body controller; the message sending form includes message No. 1, message No. 2 and message No. 3; wherein message No. 1, message No. 2 and message No. 3 correspond to Figure 2 Message A, message B and message C are shown in FIG.
[0064] The vehicle body controller is configured to: perform self-inspection and repair according to the fault type of the AI camera and determine whether to upload the fault information to the vehicle instrument according to the result of the self-inspection and repair.
[0065] Furthermore, the AI camera is connected to the vehicle controller via a CAN bus.
[0066] Furthermore, judging whether to upload the fault information to the vehicle instrument according to the result of the self-check and repair includes: setting a repair upper limit during the self-check and repair, and uploading the fault information to the vehicle instrument when and only when the number of self-check and repair exceeds the set repair upper limit.
[0067] Furthermore, different fault types correspond to different repair upper limits. Specifically: when the fault type of the AI camera is the first fault or the third fault, the corresponding repair upper limit is the upper limit of the power-on times; when the fault type of the AI camera is the second fault, the corresponding repair upper limit is the upper limit of the wiper start times.
[0068] The steps involved in the system of the above embodiment 2 correspond to those of the method embodiment 1. For the specific implementation method, please refer to the relevant description part of embodiment 1.
[0069] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.
[0070] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.
Claims
1. A self-checking and repairing method for an AI camera status for an excavator, characterized in that: include: Collect the operation data of AI cameras; The operation state of the AI camera is judged based on the obtained operation data, the sending form of the message is determined according to the obtained operation state, and the fault type of the AI camera is determined according to the sending form of the obtained message; wherein the fault type of the AI camera includes a first fault, a second fault and a third fault; According to the fault type of the AI camera, self-check and repair are performed. Specifically, when the fault type is the first fault or the third fault, the AI camera is powered on again and the number of power-on times is counted. If the number of power-on times is greater than the power-on threshold, the fault information of the fault type is uploaded to the vehicle instrument of the excavator. When the fault type is the second fault, the wiper is started and the single start time of the wiper is set, and the number of starts of the wiper is counted. If the number of starts is greater than the start threshold, the fault information of the fault type is uploaded to the vehicle instrument of the excavator.
2. The AI camera status self-check and repair method for an excavator as claimed in claim 1, characterized in that: The first fault indicates that the working state of the AI camera is abnormal, the second fault indicates that the AI camera is dirty or blocked, and the third fault indicates that the picture of the AI camera is abnormal.
3. The AI camera status self-checking and repairing method for an excavator as claimed in claim 1, characterized in that: The fault type of the AI camera is determined by the sending form of the obtained message. Specifically, the message includes three types of messages, namely, message No. 1, message No. 2 and message No. 3, and a message sending threshold is set before the message is sent.
4. The AI camera status self-checking and repairing method for an excavator as claimed in claim 3, characterized in that: When the time for not receiving message No. 1 exceeds the set message sending threshold, it is determined that the AI camera has a first fault. When the received message is in the form of message No. 2, it is determined that the AI camera has a second fault. When the received message is in the form of message No. 3, it is determined that the AI camera has a third fault.
5. The AI camera status self-checking and repairing method for an excavator as claimed in claim 4, characterized in that: Among the three fault types, the first fault has a higher priority than the second and third faults, that is, first determine whether the fault type of the AI camera is the first fault. When it is determined that it is not the first fault, then determine the second and third faults.
6. An AI camera status self-check and repair system for excavators, characterized in that: include: AI camera, vehicle controller, body controller, wipers and vehicle instruments.
7. The AI camera status self-checking and repairing system for excavators as claimed in claim 6, characterized in that: The AI camera includes a camera body and an MCU processor; The MCU processor is configured to: collect operating data of the AI camera, determine the operating status of the AI camera based on the obtained operating data, determine the sending form of the message according to the obtained operating status, and send the message to the vehicle controller; The vehicle controller is configured to: determine the fault type of the AI camera in the form of the obtained message, and send the fault information to the vehicle body controller; The vehicle body controller is configured to: perform self-inspection and repair according to the fault type of the AI camera and determine whether to upload the fault information to the vehicle instrument according to the result of the self-inspection and repair.
8. The AI camera status self-checking and repairing system for excavators as claimed in claim 7, characterized in that: The AI camera is connected to the vehicle controller via a CAN bus.
9. The AI camera status self-checking and repairing system for excavators as claimed in claim 7, characterized in that: Determine whether to upload the fault information to the vehicle instrument according to the result of self-check and repair, including: setting a repair upper limit during self-check and repair, and uploading the fault information to the vehicle instrument when and only when the number of self-check and repair exceeds the set repair upper limit.
10. The AI camera status self-checking and repairing system for excavators according to claim 9, characterized in that: Different fault types correspond to different repair limits. Specifically, when the fault type of the AI camera is the first fault or the third fault, the corresponding repair limit is the upper limit of the power-on times. When the fault type of the AI camera is the second fault, the corresponding repair upper limit is the upper limit of the wiper startup times.