Camera system anomaly detection method, device, camera system and unmanned vehicle

Abstract

The present disclosure proposes a camera system anomaly detection method, device, camera system, and unmanned vehicle, relating to the field of unmanned vehicle technology. The camera system anomaly detection method is executed by a camera system anomaly detection device, and includes: detecting the loop current of the camera module; if the cause of the camera system anomaly cannot be determined based on the detection result of the loop current of the camera module, detecting the connection status of the deserializer and the serializer; if the cause of the camera system anomaly cannot be determined based on the detection result of the connection status of the deserializer and the serializer, detecting at least one of the working status and trigger status of the camera. Through the above steps, it is possible to automatically detect key factors that affect the normal operation of the camera when a camera link fails, output the cause of the anomaly based on the detection result, accelerate the camera fault detection speed, and save problem troubleshooting time.

Description

Technical Field

[0001] The present disclosure relates to the field of unmanned driving, and in particular to a camera system anomaly detection method, device, camera system, and unmanned vehicle. Background Art

[0002] Currently, unmanned vehicles are used to automatically transport people or objects from one location to another. These vehicles use sensors to collect environmental information and perform automated transportation. Logistics transport using unmanned vehicles controlled by autonomous driving technology has greatly improved the convenience of production and life, while also saving labor costs.

[0003] Autonomous vehicles are equipped with multiple cameras, such as a traffic light recognition camera for identifying traffic lights; a front-facing surround-view camera, a left-side surround-view camera, a right-side surround-view camera, and a rear-facing surround-view camera for image recognition around the vehicle. With so many cameras in an autonomous vehicle, malfunctions can hinder its operation. Currently, camera malfunctions are typically manually investigated by maintenance personnel. Summary of the Invention

[0004] A technical problem to be solved by the present disclosure is to provide a camera system anomaly detection method, device, camera system and unmanned vehicle.

[0005] According to a first aspect of the present disclosure, a camera system abnormality detection method is proposed, wherein the camera system includes a power supply module, a deserializer, and a camera module, wherein the camera module includes a serializer and a camera; the camera system abnormality detection method is executed by a camera system abnormality detection device, and includes: detecting a loop current of the camera module; if the cause of the abnormality of the camera system cannot be determined based on the detection result of the loop current of the camera module, detecting the connection status of the deserializer and the serializer; if the cause of the abnormality of the camera system cannot be determined based on the detection result of the connection status of the deserializer and the serializer, detecting at least one of the working status and the trigger status of the camera.

[0006] In some embodiments, detecting the loop current of the camera module includes: obtaining the loop current of the camera module; when the loop current of the camera module is zero, determining that the cause of the abnormality of the camera system is a broken circuit in the camera circuit; when the loop current of the camera module is greater than the rated maximum operating current of the camera, determining that the cause of the abnormality of the camera system is a short circuit in the camera circuit.

[0007] In some embodiments, detecting the connection status of the deserializer and the serializer includes: obtaining a lock status identifier of the deserializer; and determining that the cause of the abnormality of the camera system is a failure to connect the deserializer and the serializer when the lock status identifier of the deserializer is a first value, wherein the first value indicates that the deserializer is in an unlocked state.

[0008] In some embodiments, detecting the connection status of the deserializer and the serializer further includes: when it is determined that the cause of the abnormality of the camera system is the failure of the connection between the deserializer and the serializer, performing at least one of the following detections: detecting the input power voltage of the power supply module; detecting the eye width range of the camera signal measured by the deserializer; and detecting the timing of the lock establishment request of the deserializer and the camera trigger signal.

[0009] In some embodiments, the detecting of the input power voltage of the power supply module includes: obtaining the input power voltage of the power supply module; when the input power voltage of the power supply module is less than an operating voltage threshold, determining that the reason for the failure of the connection between the deserializer and the serializer is: the input power voltage of the power supply module is too low.

[0010] In some embodiments, the detecting of the input power voltage of the power supply module further includes: obtaining waveform information of the input power voltage of the power supply module during the camera startup process; when the waveform information indicates that there is a voltage drop in the input power during the camera startup process, or the power ripple exceeds a normal range, determining that the cause of the connection failure between the deserializer and the serializer is: the input power voltage of the power supply module is unstable.

[0011] In some embodiments, detecting the camera signal eye width range measured by the deserializer includes: obtaining the camera signal eye width range measured by the deserializer; and when the camera signal eye width range is not within a normal value range, determining that a reason for the connection failure between the deserializer and the serializer is that a loop of the camera module does not meet the impedance requirements of the transmission cable.

[0012] In some embodiments, detecting the timing of the lock establishment request signal and the camera trigger signal of the deserializer includes: obtaining relative timing information of the lock establishment request signal and the camera trigger signal of the deserializer; when the relative timing information indicates that the camera trigger signal is earlier than the lock establishment request signal, determining that the cause of the connection failure between the deserializer and the serializer is: camera trigger signal interference.

[0013] In some embodiments, detecting at least one of the working status and trigger status of the camera includes: obtaining the working status identifier of the camera; when the working status identifier of the camera is abnormal, determining that the abnormal cause of the camera system is: camera failure; when the working status identifier of the camera is normal, obtaining the trigger status identifier of the camera; when the trigger status identifier of the camera is abnormal, determining that the abnormal cause of the camera system is: the camera did not receive a trigger signal.

[0014] According to a second aspect of the present disclosure, a camera system abnormality detection device is proposed, wherein the camera system includes a power supply module, a deserializer, and a camera module, the camera module includes a serializer and a camera, and the camera system abnormality detection device includes: a first detection module, configured to detect a loop current of the camera module; a second detection module, configured to detect a connection status between the deserializer and the serializer when the cause of the abnormality of the camera system cannot be determined based on a detection result of the loop current of the camera module; and a third detection module, configured to detect at least one of a working status and a trigger status of the camera when the cause of the abnormality of the camera system cannot be determined based on a detection result of the connection status between the deserializer and the serializer.

[0015] According to a third aspect of the present disclosure, a camera system abnormality detection device is provided, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute the camera system abnormality detection method as described above based on instructions stored in the memory.

[0016] According to a fourth aspect of the present disclosure, a camera system is proposed, comprising a main controller, a deserializer, a camera module and a power supply module; the main controller comprises the camera system abnormality detection device as described above; the deserializer is connected to the main controller and the camera module; the power supply module is connected to the camera module; the camera module comprises a serializer and a camera, and the serializer is connected to the deserializer and the camera, respectively.

[0017] In some embodiments, it also includes: a microcontroller, which is connected to the main controller and the power supply module respectively, and is configured to read at least one of the loop current of the camera module and the input power supply voltage information of the power supply module saved by the power supply module, and send the reading result to the main controller.

[0018] According to a fifth aspect of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the instructions are executed by a processor, the above-mentioned camera system abnormality detection method is implemented.

[0019] According to a sixth aspect of the present disclosure, an unmanned vehicle is also proposed, comprising the camera system as described above.

[0020] Other features and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0022] The present disclosure can be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0023] Figure 1 is a schematic structural diagram of a camera system according to some embodiments of the present disclosure;

[0024] Figure 2 1 is a flowchart of a camera system abnormality detection method according to some embodiments of the present disclosure;

[0025] Figure 3 is a schematic diagram of a current detection process according to some embodiments of the present disclosure;

[0026] Figure 4 1 is a flow chart of detecting the connection status of a deserializer and a serializer according to some embodiments of the present disclosure;

[0027] Figure 5 1 is a flowchart of camera working state and trigger state detection according to some embodiments of the present disclosure;

[0028] Figure 6 is a schematic structural diagram of a camera system abnormality detection device according to some embodiments of the present disclosure;

[0029] Figure 7 is a schematic structural diagram of a camera system according to some other embodiments of the present disclosure;

[0030] Figure 8 is a schematic structural diagram of a camera system abnormality detection device according to some embodiments of the present disclosure;

[0031] Figure 9 Schematic diagram of the structure of a computer system according to some embodiments of the present disclosure.

[0032] Figure 10 Schematic diagram of the structure of an unmanned vehicle according to some embodiments of the present disclosure.

[0033] Figure 11 Schematic diagram of the three-dimensional structure of an unmanned vehicle according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0034] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure.

[0035] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0036] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0037] Technologies, methods and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, such technologies, methods and equipment should be considered part of the authorization specification.

[0038] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0039] It should be noted that like reference numerals and letters refer to like items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0040] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0041] Figure 1 FIG. 1 is a schematic diagram of the structure of a camera system according to some embodiments of the present disclosure. Figure 1 As shown, the camera system according to the embodiment of the present disclosure includes: a main controller 110 , a deserializer 120 , a camera module 130 , and a power supply module 140 .

[0042] The main controller 110 is connected to the deserializer 120 .

[0043] In some embodiments, the main controller 110 includes a camera system abnormality detection device, wherein the camera system abnormality detection device is configured to detect abnormal causes of the camera system.

[0044] For example, when the main controller 110 cannot obtain the data output by the camera, the camera system abnormality detection device is called to check the cause of the abnormality of the camera system.

[0045] The deserializer 120 is connected to the main controller 110 and the camera module 130. The deserializer is used to convert received serial data into parallel data.

[0046] The camera module 130 includes a serializer 131 and a camera 132, wherein the serializer 131 is connected to the deserializer 120 and the camera 132 respectively. The serializer 131 is used to convert received parallel data into serial data.

[0047] Exemplarily, when the serializer 131 receives image data output by the camera 132, it converts the parallel image data into serial image data, and then transmits the serial image data to the deserializer 120. After the deserializer 120 converts the serial image data into parallel image data, it transmits the parallel image data to the main controller 110.

[0048] The power supply module 140 is connected to the camera module 130 and is used to supply power to the camera module 130 .

[0049] In some embodiments, the camera system further includes: a microcontroller, connected to the main controller and the power supply module respectively, configured to read at least one of the loop current of the camera module and the input power supply voltage information of the power supply module stored in the power supply module, and send the reading result to the main controller.

[0050] In the disclosed embodiment, the above camera system can automatically detect key factors that affect the normal operation of the camera when a camera link fails, output the cause of the abnormality based on the detection results, accelerate the camera fault detection speed, and save problem troubleshooting time.

[0051] Figure 2 FIG. 1 is a flow chart of a camera system abnormality detection method according to some embodiments of the present disclosure. In the embodiments of the present disclosure, the camera system abnormality detection method may be executed by a camera system abnormality detection device. Figure 2 As shown, the camera system abnormality detection method of the embodiment of the present disclosure includes:

[0052] Step S210: detecting the loop current of the camera module.

[0053] In some embodiments, when the main controller of the unmanned vehicle cannot obtain camera data during operation, the camera system abnormality detection process of the embodiment of the present disclosure is executed.

[0054] In some embodiments, step S210 includes reading the camera circuit current stored in a register within the power supply module and determining whether the current is within a normal range. If the current is not within the normal range, the cause of the camera system anomaly can be determined based on the current. If the current is within the normal range, it indicates that the electrical circuit where the camera is located is normal, and the cause of the camera system anomaly cannot be determined based on the current, requiring further investigation.

[0055] In other embodiments, step S210 adopts Figure 3 The process shown is for current detection.

[0056] Step S220 : If the cause of the abnormality of the camera system cannot be determined based on the detection result of the loop current of the camera module, the connection status of the deserializer and the serializer is detected.

[0057] In some embodiments, step S220 includes: obtaining a lock status identifier of the deserializer; if the lock status identifier of the deserializer is a first value, determining that the cause of the camera system abnormality is a failure to connect the deserializer and the serializer; if the lock status identifier of the deserializer is a second value, determining that the connection between the deserializer and the serializer is successful, and that the cause of the camera system abnormality cannot be determined based on the connection status of the deserializer and the serializer, and further troubleshooting is required. The first value indicates that the deserializer is in an unlocked state, which indicates that the connection between the serializer and the deserializer has failed, and the second value indicates that the deserializer is in a locked state, which indicates that the connection between the serializer and the deserializer is successful.

[0058] In other embodiments, step S220 further includes: after determining that the cause of the camera system anomaly is a failure to connect the deserializer and serializer, performing at least one of the following tests to clarify the cause of the failure: testing the input power voltage of the power supply module; testing the camera signal eye width range measured by the deserializer; and testing the timing of the deserializer lock establishment request and the camera trigger signal. Through the above-mentioned processing, the cause of the camera system anomaly can be accurately and layer by layer identified, thereby improving the efficiency of anomaly detection.

[0059] In some further embodiments, step S220 adopts Figure 4 The process shown is used to detect the connection status of the deserializer and the serializer.

[0060] Step S230: If the cause of the abnormality of the camera system cannot be determined based on the detection result of the connection status of the deserializer and the serializer, at least one of the working status and the triggering status of the camera is detected.

[0061] In some embodiments, step S230 uses Figure 5 The process shown is to detect the working status and trigger status of the camera.

[0062] In some embodiments, if the cause of the camera system abnormality cannot be determined based on the camera's operating status and trigger status, a prompt message indicating that the abnormality cause has failed to be determined is output. For example, the following prompt message is output: The camera failed to start due to an unknown reason, and professional intervention is required.

[0063] In the disclosed embodiment, the above steps can automatically detect key factors that affect the normal operation of the camera when a camera link fails, output the abnormality cause based on the detection result, accelerate the camera fault detection speed, and save problem troubleshooting time.

[0064] Figure 3 FIG. 1 is a flow chart of current detection according to some embodiments of the present disclosure. Figure 3 As shown in Figure 1, the current detection process includes:

[0065] Step S211: Obtain the loop current of the camera module.

[0066] In some embodiments, the current of the camera circuit stored in the internal register of the power supply module is read.

[0067] Step S212: Determine whether the loop current is 0.

[0068] When the loop current is 0, step S213 is executed; when the loop current is not 0, step S214 is executed.

[0069] Step S213: Determine that the abnormality of the camera system is caused by a broken circuit in the camera circuit.

[0070] If the camera loop current is 0, it indicates that the camera system abnormality is caused by a camera circuit break, for example, a circuit break caused by a camera circuit hardware fault such as a loose connector or a disconnected electrical circuit.

[0071] In some embodiments, after step S213, the process further includes: outputting the cause of the camera system abnormality, for example, outputting the following cause of the camera system abnormality: camera circuit hardware failure, loose connector, or disconnected electrical circuit.

[0072] Step S214: Determine whether the loop current is greater than I0.

[0073] Where I0 is the rated maximum operating current of the camera. In this step, the loop current of the camera is compared with the rated maximum operating current of the camera.

[0074] When the loop current of the camera is greater than I0, step S215 is executed; when the loop current of the camera is less than or equal to I0, step S220 is executed.

[0075] Step S215: Determine that the abnormality of the camera system is caused by a short circuit in the camera circuit.

[0076] If the camera loop current is greater than the rated maximum operating current, it indicates that the camera system abnormality is caused by a short circuit in the camera's electrical circuit.

[0077] In some embodiments, after step S215 , the method further includes: outputting a camera system abnormality cause, for example, outputting the following camera system abnormality cause: a short circuit in the camera electrical circuit.

[0078] In the embodiment of the present disclosure, the above steps are used to implement the automatic detection process of the camera loop current.

[0079] Figure 4 FIG. 1 is a flow chart of detecting the connection status of a deserializer and a serializer according to some embodiments of the present disclosure. Figure 4 As shown, the connection status detection process between the deserializer and the serializer in the embodiment of the present disclosure includes:

[0080] Step S221: Obtain the lock status identifier of the deserializer.

[0081] In some embodiments, a lock status flag stored in an internal register of the deserializer is read. The lock status flag includes two possible values: a first value and a second value. The first value indicates that the deserializer is in an unlocked state, which indicates that the connection between the serializer and the deserializer has failed, and the second value indicates that the deserializer is in a locked state, which indicates that the connection between the serializer and the deserializer is good.

[0082] When the lock state identifier of the deserializer is the first value, step S222 is executed; when the lock state identifier of the deserializer is the second value, step S230 is executed.

[0083] Step S222: Obtain the input power voltage of the power supply module.

[0084] In some embodiments, the voltage value of the input power supply is obtained by a voltage sampling circuit. The input power supply voltage value is then compared with the minimum input voltage for the power supply module to operate normally. If the input power supply voltage of the power supply module is less than the minimum input voltage U1 for the power supply module to operate normally, step S223 is executed; if the input power supply voltage of the power supply module is greater than or equal to U1, step S224 is executed.

[0085] Exemplarily, the power supply module adopts a step-down conversion circuit such as a BUCK circuit. In this example, the minimum input voltage U1 for the BUCK circuit to work properly is determined based on the desired output voltage of the BUCK circuit and the minimum voltage difference of the BUCK circuit, and then the input power supply voltage is compared with U1. For example, if the desired output voltage is 9V and the minimum voltage difference is 1.1V, the minimum input voltage for the BUCK circuit to work properly is 10.1V. Assuming that the input power supply voltage is 12V, since the input power supply voltage is greater than U1, the power supply module can work stably; assuming that the input power supply voltage is 9.5V, since the input power supply voltage is less than U1, the power supply module cannot work stably, and thus the power supply frequency does not meet the requirements of the Gigabit Multimedia Serial Link (GMSL) used by the camera, resulting in abnormal camera startup.

[0086] Step S223: Determine that the cause of the camera system abnormality is: the input power voltage of the power supply module is too low, resulting in a connection failure between the deserializer and the serializer.

[0087] In some embodiments, after step S223 , the cause of the camera system abnormality is output.

[0088] Step S224: Obtain the camera signal eye width range measured by the deserializer.

[0089] After obtaining the camera signal eye width range measured by the deserializer, compare the camera signal eye width range with the normal range. For example, the normal range of the camera signal eye width range is 60% to 80%. If the camera signal eye width range measured by the deserializer is between 60% and 80%, the camera signal eye width range is normal. If the camera signal eye width range measured by the deserializer is not between 60% and 80%, the camera signal eye width range is abnormal.

[0090] If the camera signal eye width range is abnormal, step S225 is executed; if the camera signal eye width range is normal, step S226 is executed.

[0091] Step S225: Determine that the cause of the camera system abnormality is: the loop of the camera module does not meet the impedance requirement of the transmission cable, resulting in a connection failure between the deserializer and the serializer.

[0092] In some embodiments, after step S225 , the cause of the camera system abnormality is output.

[0093] Step S226: Obtain the relative timing of the deserializer lock establishment request and the camera trigger signal.

[0094] After step S226, it is determined whether the camera trigger signal is earlier than the lock establishment request based on the relative time. If the camera trigger signal is earlier than the lock establishment request, step S227 is executed; if the camera trigger signal is not earlier than the lock establishment request, step S228 is executed.

[0095] Step S227: Determine that the cause of the camera system abnormality is: interference with the camera trigger signal causes the deserializer and serializer to fail to connect.

[0096] If the camera trigger signal is received earlier than the lock establishment request, it indicates that the camera abnormality is caused by the trigger signal received by the camera affecting the normal establishment of the GMSL link lock, resulting in the failure of the deserializer and serializer connection.

[0097] In some embodiments, after step S227 , the cause of the camera system abnormality is output.

[0098] Step S228: Acquire waveform information of the input power voltage of the power supply module during the camera startup process.

[0099] If the input power voltage has a voltage drop or an abnormal power ripple, step S229 is executed; if the input power voltage has no voltage drop or abnormal power ripple, step S240 is executed.

[0100] Step S229: Determine that the cause of the camera system abnormality is: unstable input power causing the deserializer and serializer to fail to connect.

[0101] If there is a voltage drop when the camera is powered on, the camera abnormality is caused by unstable power supply at the moment of power-on, resulting in a connection failure between the deserializer and serializer, which in turn causes the camera to fail to start. If the power ripple exceeds the normal range when the camera starts, the camera abnormality is caused by unstable power supply, resulting in a connection failure between the deserializer and serializer, which in turn causes the camera to fail to start.

[0102] Step S240: Determine the unknown cause.

[0103] In some embodiments, if the cause of the camera system abnormality cannot be determined based on the connection status of the deserializer and the serializer, a prompt message indicating that the abnormality cause has failed to be determined is output. For example, the following prompt message is output: The camera failed to start due to an unknown reason and professional intervention is required.

[0104] In the disclosed embodiment, the above steps realize automatic detection of the connection status of the deserializer and the serializer. By accurately troubleshooting the cause of the camera abnormality layer by layer, the efficiency of troubleshooting the cause of the camera abnormality can be improved and the cause of the camera abnormality can be accurately located.

[0105] Figure 5 FIG. 1 is a flow chart of camera working state and trigger state detection according to some embodiments of the present disclosure. Figure 5 As shown, the camera working status and trigger status detection process includes:

[0106] Step S231: Obtain the camera's working status identifier.

[0107] In some embodiments, the working status identifier of the camera stored in the internal register of the serializer is read.

[0108] Step S232: Determine whether the working status indicator is abnormal.

[0109] In some embodiments, when the working status identifier of the camera is a value indicating that it is in a normal working state, it is determined that the working status identifier is normal; otherwise, it is determined that the working status identifier is abnormal.

[0110] If it is determined that the camera's working status identifier is abnormal, step S233 is executed; if it is determined that the camera's working status identifier is normal, step S234 is executed.

[0111] Step S233: Determine that the abnormal cause of the camera system is: camera failure.

[0112] In some embodiments, after step S233, the abnormality reason of the camera system is output, for example, the following information is output: the camera is faulty and needs to be replaced.

[0113] Step S234: Obtain the trigger status identifier of the camera.

[0114] In some embodiments, the trigger status identifier of the camera stored in the register is read.

[0115] Step S235: Determine whether the trigger status flag is abnormal.

[0116] In some embodiments, when the working status identifier of the camera is a value indicating that it is in a triggered state, it is determined that the trigger status identifier is normal; otherwise, it is determined that the trigger status identifier is abnormal.

[0117] If the trigger status flag is abnormal, execute step S236; if the trigger status flag is normal, execute step S240.

[0118] Step S236: Determine that the abnormality of the camera system is caused by the camera not receiving a trigger signal.

[0119] In some embodiments, after step S236 , the abnormality reason of the camera system is output, for example, the following information is output: the camera did not receive a trigger signal, and no data is output.

[0120] Step S240: Determine the unknown cause.

[0121] In some embodiments, if the cause of the camera system abnormality cannot be determined based on the camera's operating status and trigger status, a prompt message indicating that the abnormality cause has failed to be determined is output. For example, the following prompt message is output: The camera failed to start due to an unknown reason, and professional intervention is required.

[0122] In the embodiment of the present disclosure, the automatic detection of the camera working state and triggering state is achieved through the above steps.

[0123] Figure 6 FIG. 1 is a schematic diagram of the structure of a camera system abnormality detection device according to some embodiments of the present disclosure. Figure 6 As shown, the camera system abnormality detection device 600 of the embodiment of the present disclosure includes: a first detection module 610, a second detection module 620, and a third detection module 630.

[0124] The first detection module 610 is configured to detect the loop current of the camera module.

[0125] In some embodiments, the first detection module 610 detects the loop current of the camera module, including: obtaining the loop current of the camera module; when the loop current of the camera module is zero, determining that the cause of the abnormality of the camera system is a broken circuit in the camera circuit; when the loop current of the camera module is greater than the rated maximum operating current of the camera, determining that the cause of the abnormality of the camera system is a short circuit in the camera circuit.

[0126] The second detection module 620 is configured to detect the connection status of the deserializer and the serializer when the cause of the abnormality of the camera system cannot be determined based on the detection result of the loop current of the camera module.

[0127] In some embodiments, the second detection module 620 detects the connection status of the deserializer and the serializer, including: obtaining a lock status identifier of the deserializer; when the lock status identifier of the deserializer is a first value, determining that the cause of the abnormality of the camera system is a failure to connect the deserializer and the serializer, wherein the first value indicates that the deserializer is in an unlocked state.

[0128] In other embodiments, the second detection module 620 detects the connection status of the deserializer and serializer by: obtaining a lock status identifier of the deserializer; and when the lock status identifier of the deserializer is a first value, performing at least one of the following tests: detecting the input power voltage of the power supply module; detecting the camera signal eye width range measured by the deserializer; and detecting the timing of the deserializer lock establishment request and the camera trigger signal. By performing at least one of the above tests after determining that the cause of the camera system anomaly is a failure to connect the deserializer and serializer, the cause of the failure can be further clarified, enabling layer-by-layer and precise troubleshooting of the cause of the camera system anomaly, thereby improving the efficiency of anomaly troubleshooting.

[0129] The third detection module 630 is configured to detect at least one of the working state and the triggering state of the camera when the cause of the abnormality of the camera system cannot be determined according to the detection result of the connection state of the deserializer and the serializer.

[0130] In some embodiments, the third detection module 630 obtains the camera's working status identifier; when the camera's working status identifier is abnormal, the abnormal cause of the camera system is determined to be: camera failure; when the camera's working status identifier is normal, the camera's trigger status identifier is obtained; when the camera's trigger status identifier is abnormal, the abnormal cause of the camera system is determined to be: the camera did not receive a trigger signal.

[0131] In the disclosed embodiment, the above device can automatically detect key factors that affect the normal operation of the camera when the camera system fails, output the abnormality cause according to the detection result, accelerate the camera fault detection speed, and save problem troubleshooting time.

[0132] Figure 7 FIG. 1 is a schematic structural diagram of a camera system according to some other embodiments of the present disclosure. Figure 7 As shown, the camera system of the embodiment of the present disclosure includes: a power supply module 710 , a camera module 720 , a main controller 730 , a deserializer 740 , and a microcontroller 750 .

[0133] The power supply module 710 includes an input power supply 711 , a BUCK circuit 712 , and a load switch.

[0134] In the power supply module 710, the input power supply 710 passes through the BUCK circuit 712 composed of a step-down converter, stabilizes the voltage at 9V and inputs it to the load switch. After passing through the load switch, the voltage is loaded onto the 50Ω coaxial cable through the filter circuit composed of the L2 inductor, thereby powering the camera module.

[0135] The camera module 720 includes a serializer 721 , a camera 722 , a filter circuit 723 , and a power supply voltage regulator module 724 .

[0136] In the camera module 720 , the input voltage of the power supply module 710 is supplied to the serializer 721 and the camera 722 (illustratively, the camera includes an image sensor and an optical lens control circuit) after passing through the filter circuit 723 and the power supply voltage regulator module 724 .

[0137] The deserializer 740 is connected to the main controller 730 and the camera module 720. The deserializer 740 is used to convert received serial data into parallel data.

[0138] For the image data captured by the camera, after the parallel data is compressed into serial data through the serializer, it is transmitted on a 50Ω coaxial cable and then transmitted to the deserializer through two AC coupling capacitors C3 and C4. The deserializer converts the serial data into parallel data and transmits the image data to the main controller in the form of Mobile Industry Processor Interface (MIPI) signals.

[0139] The main controller 730 can configure the deserializer and the serializer via a two-wire serial bus (Inter-Integrated Circuit, I2C for short).

[0140] The microcontroller 750 and the main controller 730 can communicate via a serial port, and the microcontroller 750 can communicate with the load switch via I2C. For example, the microcontroller 750 can read the loop current of the camera module stored by the load switch via I2C.

[0141] In some embodiments, the main controller 730 includes a camera system abnormality detection device, wherein the camera system abnormality detection device is configured to detect abnormal causes of the camera system.

[0142] In the disclosed embodiment, the above camera system can automatically detect key factors that affect the normal operation of the camera when the camera system fails, output the cause of the abnormality based on the detection results, accelerate the camera failure detection speed, and save problem troubleshooting time.

[0143] Figure 8 Schematic diagram of the structure of a camera system abnormality detection device according to some embodiments of the present disclosure.

[0144] like Figure 8 As shown, camera system anomaly detection apparatus 800 includes a memory 810 and a processor 820 coupled to the memory 810. The memory 810 is configured to store instructions for executing the camera system anomaly detection method according to the embodiments. The processor 820 is configured to execute the camera system anomaly detection method according to any of the embodiments of the present disclosure based on the instructions stored in the memory 810.

[0145] Figure 9 Schematic diagram of the structure of a computer system according to some embodiments of the present disclosure.

[0146] like Figure 9 As shown, computer system 900 may be implemented as a general-purpose computing device. Computer system 900 includes memory 910, processor 920, and bus 930 that connects various system components.

[0147] Memory 910 may include, for example, system memory, non-volatile storage media, and the like. System memory, for example, stores an operating system, application programs, a boot loader, and other programs. System memory may include volatile storage media, such as random access memory (RAM) and / or cache memory. Non-volatile storage media, for example, stores instructions for executing at least one embodiment of the camera system anomaly detection method. Non-volatile storage media include, but are not limited to, disk storage, optical storage, and flash memory.

[0148] The processor 920 can be implemented as a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, or as discrete hardware components such as discrete gates or transistors. Accordingly, each module, such as the first detection module, the second detection module, and the third detection module, can be implemented by a central processing unit (CPU) executing instructions in a memory for executing corresponding steps, or by dedicated circuits for executing corresponding steps.

[0149] The bus 930 may use any of a variety of bus architectures, including, but not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, and a Peripheral Component Interconnect (PCI) bus.

[0150] The computer system 900 interfaces 940, 950, and 960, as well as the memory 910 and the processor 920, can be connected via a bus 930. The input / output interface 940 provides a connection interface for input / output devices such as a display, mouse, and keyboard. The network interface 950 provides a connection interface for various networked devices. The storage interface 960 provides a connection interface for external storage devices such as floppy disks, USB flash drives, and SD cards.

[0151] Figure 10 Schematic diagram of the structure of an unmanned vehicle according to some embodiments of the present disclosure. Figure 11 The following is a schematic diagram of the three-dimensional structure of an unmanned vehicle according to some embodiments of the present disclosure. Figure 10 and Figure 11 The unmanned vehicle provided in the embodiment of the present disclosure is described.

[0152] As attached Figure 10 As shown, the unmanned vehicle includes four parts: a chassis module 1010, an automatic driving module 1020, a cargo box module 1030 and a remote monitoring and streaming module 1040.

[0153] In some embodiments, the chassis module 1010 primarily includes a battery, a power management device, a chassis controller, a motor driver, and a power motor. The battery provides power for the entire autonomous vehicle system. The power management device converts the battery output into different voltage levels for each functional module and controls power on and off. The chassis controller receives motion commands from the autonomous driving module and controls the autonomous vehicle's steering, forward movement, reverse movement, braking, and other functions.

[0154] In some embodiments, the autonomous driving module 1020 includes a core processing unit (Orin or Xavier module), a traffic light recognition camera, front, rear, and left / right surround view cameras, a multi-line lidar, a positioning module (such as Beidou or GPS), and an inertial navigation unit. The cameras and the autonomous driving module can communicate using a GMSL link to increase transmission speed and reduce wiring.

[0155] In some embodiments, the autonomous driving module 1020 includes the camera system of the above-described embodiments.

[0156] In some embodiments, the remote monitoring streaming module 1030 is composed of a front surveillance camera, a rear surveillance camera, a left surveillance camera, a right surveillance camera, and a streaming module. This module transmits video data collected by the surveillance cameras to a backend server for viewing by backend operators. The wireless communication module communicates with the backend server via an antenna, enabling backend operators to remotely control the unmanned vehicle.

[0157] The cargo box module 1040 is the cargo-carrying device of the unmanned vehicle. In some embodiments, the cargo box module 1040 is also equipped with a display and interaction module, which is used for interaction between the unmanned vehicle and the user. Users can use the display and interaction module to perform operations such as picking up items, storing items, and purchasing items. The type of cargo box can be changed according to actual needs. For example, in logistics scenarios, the cargo box can include multiple sub-boxes of different sizes, which can be used to load goods for delivery. In retail scenarios, the cargo box can be configured as a transparent box to allow users to directly see the products for sale.

[0158] The unmanned vehicle of the disclosed embodiment can automatically detect key factors that affect the normal operation of the camera when a camera link fails, output the cause of the abnormality based on the detection results, accelerate the camera fault detection speed, and save problem troubleshooting time.

[0159] Here, various aspects of the present disclosure are described with reference to flowcharts and / or block diagrams of methods, devices, and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks, can be implemented by computer-readable program instructions.

[0160] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable device to produce a machine, so that the processor executes the instructions to produce means for implementing the functions specified in one or more blocks in the flowcharts and / or block diagrams.

[0161] These computer-readable program instructions may also be stored in a computer-readable memory, which cause the computer to operate in a specific manner to produce an article of manufacture, including instructions for implementing the functions specified in one or more blocks in the flowcharts and / or block diagrams.

[0162] The present disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects.

[0163] The camera system abnormality detection method, device, camera system and unmanned vehicle in the above-mentioned embodiments can automatically detect key factors affecting the normal operation of the camera when a camera link fails, output the cause of the abnormality based on the detection results, accelerate the camera fault detection speed, and save problem troubleshooting time.

[0164] The camera system anomaly detection method, apparatus, camera system, and unmanned vehicle according to the present disclosure have been described in detail. To avoid obscuring the underlying principles of the present disclosure, some details known in the art have been omitted. Based on the above description, those skilled in the art will fully understand how to implement the technical solutions disclosed herein.

Claims

1. A camera system anomaly detection method, wherein the camera system includes a power supply module, a deserializer, and a camera module, wherein the camera module includes a serializer and a camera; the camera system anomaly detection method is performed by a camera system anomaly detection device, comprising: detecting a loop current of the camera module; When the cause of the abnormality of the camera system cannot be determined based on the detection result of the loop current of the camera module, the connection status of the deserializer and the serializer is detected, including: obtaining a lock status identifier of the deserializer; when the lock status identifier of the deserializer is a first value, determining that the cause of the abnormality of the camera system is a failure to connect the deserializer and the serializer, wherein the first value indicates that the deserializer is in an unlocked state; when it is determined that the cause of the abnormality of the camera system is a failure to connect the deserializer and the serializer, performing at least one of the following tests: detecting the input power supply voltage of the power supply module; detecting the camera signal eye width range measured by the deserializer; and detecting the timing of the lock establishment request of the deserializer and the camera trigger signal. When the cause of the abnormality of the camera system cannot be determined according to the detection result of the connection status of the deserializer and the serializer, at least one of the working status and the triggering status of the camera is detected.

2. The camera system abnormality detection method according to claim 1, wherein: Detecting the loop current of the camera module includes: Obtaining a loop current of the camera module; When the loop current of the camera module is zero, determining that the abnormality of the camera system is caused by a broken circuit of the camera circuit; When the loop current of the camera module is greater than the rated maximum operating current of the camera, it is determined that the abnormality of the camera system is caused by a short circuit in the camera circuit.

3. The camera system abnormality detection method according to claim 1, wherein: The detecting the input power voltage of the power supply module includes: Obtaining the input power voltage of the power supply module; When the input power voltage of the power supply module is less than the operating voltage threshold, it is determined that a reason for the failure of the connection between the deserializer and the serializer is that the input power voltage of the power supply module is too low.

4. The camera system abnormality detection method according to claim 3, wherein: The detecting of the input power supply voltage of the power supply module further includes: Acquiring waveform information of an input power voltage of the power supply module during startup of the camera; When the waveform information indicates that the input power has a voltage drop during camera startup, or the power ripple exceeds a normal range, it is determined that the reason for the failure to connect the deserializer and the serializer is that the input power voltage of the power supply module is unstable.

5. The camera system abnormality detection method according to claim 1, wherein: Detecting the eye width range of the camera signal measured by the deserializer includes: Obtaining an eye width range of a camera signal measured by the deserializer; When the camera signal eye width range is not within a normal value range, it is determined that the reason for the failure in connecting the deserializer and the serializer is that a loop of the camera module does not meet the impedance requirement of the transmission cable.

6. The camera system abnormality detection method according to claim 1, wherein: Detecting the timing of the lock establishment request signal of the deserializer and the camera trigger signal includes: Obtain the relative timing information of the deserializer's lock establishment request signal and the camera's trigger signal; When the relative timing information indicates that the camera trigger signal is earlier than the lock establishment request signal, it is determined that the reason for the failure of the connection between the deserializer and the serializer is: interference of the camera trigger signal.

7. The camera system abnormality detection method according to claim 1, wherein: Detecting at least one of the working state and the triggering state of the camera includes: Obtaining a working status identifier of the camera; When the working status indicator of the camera is abnormal, determining that the abnormal cause of the camera system is: camera failure; When the working status identifier of the camera is normal, obtaining the trigger status identifier of the camera; When the trigger status indicator of the camera is abnormal, it is determined that the abnormal cause of the camera system is: the camera does not receive a trigger signal.

8. A camera system abnormality detection device, the camera system comprising a power supply module, a deserializer, and a camera module, the camera module comprising a serializer and a camera, the camera system abnormality detection device comprising: a first detection module, configured to detect a loop current of the camera module; The second detection module is configured to detect the connection status of the deserializer and the serializer when the cause of the abnormality of the camera system cannot be determined according to the detection result of the loop current of the camera module, including: obtaining a lock status identifier of the deserializer; when the lock status identifier of the deserializer is a first value, determining that the cause of the abnormality of the camera system is a failure to connect the deserializer and the serializer, wherein the first value indicates that the deserializer is in an unlocked state; when it is determined that the cause of the abnormality of the camera system is a failure to connect the deserializer and the serializer, performing at least one of the following detections: detecting the input power supply voltage of the power supply module; detecting the camera signal eye width range measured by the deserializer; and detecting the timing of the lock establishment request of the deserializer and the camera trigger signal. The third detection module is configured to detect at least one of the working state and the trigger state of the camera when the cause of the abnormality of the camera system cannot be determined according to the detection result of the connection state of the deserializer and the serializer.

9. A camera system anomaly detection device, comprising: Memory; as well as A processor coupled to the memory, wherein the processor is configured to execute the camera system abnormality detection method according to any one of claims 1 to 7 based on instructions stored in the memory.

10. A camera system comprising a main controller, a deserializer, a camera module and a power supply module; The main controller includes the camera system abnormality detection device according to claim 8 or 9; The deserializer is connected to the main controller and the camera module; The power supply module is connected to the camera module; The camera module includes a serializer and a camera, and the serializer is connected to the deserializer and the camera respectively.

11. The camera system of claim 10, further comprising: The microcontroller is connected to the main controller and the power supply module respectively, and is configured to read at least one of the loop current of the camera module and the input power voltage information of the power supply module stored in the power supply module, and send the reading result to the main controller.

12. A computer-readable storage medium having computer program instructions stored thereon, wherein when the instructions are executed by a processor, the camera system abnormality detection method according to any one of claims 1 to 7 is implemented.

13. An unmanned vehicle comprising: A camera system as claimed in claim 10 or 11.

Images