Active repairing method and device for cameras in vehicle, medium and electronic equipment

Abstract

The embodiment of the invention provides an active repair method and device for each camera in a vehicle, a medium and electronic equipment. Acquiring camera links of the plurality of cameras relative to the deserializer; defining an interrupt portion of the camera based on the camera link and the connection state of the camera, and defining a second portion for the interrupt portion of the camera so as to trigger a hot plug detection logic based on the second portion, and defining an insertion state of the camera according to the hot plug detection logic, therefore, the insertion state of the camera is managed and controlled in the radio frequency communication link corresponding to the camera and the state of the video stream, the accuracy of the insertion state of the camera is ensured, and meanwhile, the recovery of the video stream is triggered based on the insertion state of the camera; in the recovery process of the video stream, the recovery progress of the video stream is monitored, and the recovery progress of each camera is defined until the recovery of each camera is completed, so that the insertion state of the camera and the recovery of the video stream are completed, and the self-recovery of each camera along with the dynamic movement of the vehicle is ensured.

Description

Technical Field

[0001] The present application relates to the technical field of camera repair, and more specifically, to an active repair method, device, medium and electronic device for each camera in a vehicle. Background Art

[0002] With the development of technology, the current car autonomous driving domain controller or smart cockpit needs to connect 4 to 10 cameras to sense changes in the surrounding environment. Generally, GMSL coaxial cables and serial deserializers are used to aggregate data from multiple cameras and send it to the CPU. When configuring the serial deserializer and camera link, it is generally only configured once during power-on initialization. After initialization, the video stream data is processed and the link status is no longer detected or restored. In the existing method, once a camera link abnormality occurs during the local power-on cycle, it is impossible to control the camera insertion status and the recovery of the video stream, and it is impossible to respond to the interruption of each camera in time. Summary of the invention

[0003] The embodiment of the present application provides a method, device, medium and electronic device for active repair of each camera in a vehicle. Multiple cameras are arranged on the periphery of the vehicle, and synchronously collect the surrounding environment of the vehicle as the vehicle travels; collect the camera links of multiple cameras relative to the deserializer; define the interruption part of the camera based on the camera link and the connection status of the camera, and define the second part for the interruption part of the camera, so as to trigger the hot plug detection logic based on the second part, and define the insertion state of the camera according to the hot plug detection logic, so as to control the insertion state of the camera under the state of the radio frequency communication link and video stream corresponding to the camera, ensure the accuracy of the insertion state of the camera, and at the same time, trigger the recovery of the video stream based on the insertion state of the camera; in the process of video stream recovery, monitor the recovery progress of the video stream, and define the recovery progress of each camera until the recovery of each camera is completed, so as to complete the recovery of the insertion state of the camera and the video stream, ensure that each camera can achieve self-repair with the dynamic movement of the vehicle, and can respond to the interruption of each camera in time, increase the robustness of the system and user experience.

[0004] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0005] According to one aspect of an embodiment of the present application, a method for actively repairing each camera in a vehicle is provided, which is applied to an active repair scenario of a camera; the method for actively repairing each camera in a vehicle comprises:

[0006] Multiple cameras are arranged around the vehicle and synchronously collect the surrounding environment of the vehicle as the vehicle travels; the camera links of the multiple cameras relative to the deserializer are collected;

[0007] Defining an interrupt portion of the camera based on the camera link and the connection status of the camera;

[0008] Divide the first part and the second part based on the interrupt part of the camera, and trigger the hot plug detection logic based on the second part;

[0009] In the hot-plug detection logic, the camera insertion status is defined based on the status of the RF communication link and video stream corresponding to the camera;

[0010] Triggering the resumption of video streaming based on the camera's insertion status;

[0011] During the recovery process of the video stream, monitor the recovery progress of the video stream and define the recovery progress of each camera until the recovery of each camera is completed.

[0012] Optionally, the multiple cameras are arranged on the periphery of the vehicle and synchronously collect the surrounding environment of the vehicle as the vehicle travels; collecting the camera links of the multiple cameras relative to the deserializer includes:

[0013] Collect a three-dimensional model of the vehicle;

[0014] An outer contour of the vehicle is defined based on a three-dimensional model of the vehicle, and a detection range is defined according to the outer contour of the vehicle and a moving speed of the vehicle;

[0015] Defining installation positions of multiple cameras based on the detection range and the outer contour of the vehicle, and arranging corresponding cameras along the installation positions of the multiple cameras, so that the multiple cameras are arranged around the vehicle, and the multiple cameras synchronously collect the surrounding environment of the vehicle as the vehicle travels;

[0016] Locate multiple cameras and deserializers, and synchronously control multiple cameras based on the deserializers;

[0017] Camera links of the multiple cameras relative to the deserializer are matched based on response paths of the deserializer to the multiple cameras, so as to complete the acquisition of the camera links of the multiple cameras relative to the deserializer.

[0018] Optionally, the step of defining the interruption part of the camera based on the camera link and the connection state of the camera includes:

[0019] Get the camera's connection status;

[0020] Associate the camera link and the camera's connection status;

[0021] Defining abnormal locations based on camera links and camera connection status;

[0022] Match the corresponding abnormal path according to the abnormal location;

[0023] Define the interruption section of the camera based on the abnormal path and the connection status of the camera.

[0024] Optionally, dividing the camera-based interruption part into a first part and a second part, and triggering the hot plug detection logic based on the second part, includes:

[0025] freeze-frame camera interruptions;

[0026] Dividing the first part and the second part according to the interruption part of the camera and the interruption node of the camera;

[0027] Based on the first part, the execution logic is triggered, and a flag bit indicating that the execution is completed is marked in the execution logic;

[0028] The second part is triggered based on the response of the execution completion flag bit, and at this time, the second part is adjusted from the sleep state to the working state;

[0029] When the second part is in the working state, the corresponding operation content is executed based on the second part, and the hot plug detection logic is triggered.

[0030] Optionally, in the hot plug detection logic, defining the insertion status of the camera based on the status of the radio frequency communication link and the video stream corresponding to the camera includes:

[0031] In the hot-plug detection logic, the RF communication link corresponding to the camera is collected;

[0032] Traverse the radio frequency communication link corresponding to the camera and define the status of the radio frequency communication link;

[0033] defining a first state parameter based on a state of the radio frequency communication link;

[0034] Collect video streams;

[0035] Defining an interrupt node of the video stream based on the traversal of the video stream;

[0036] Defining an interruption state of the video stream according to an interruption node of the video stream, and defining a second state parameter;

[0037] The second state parameter and the first state parameter are associated, and the insertion state of the camera is defined based on the second state parameter and the first state parameter.

[0038] Optionally, the triggering of the recovery of the video stream based on the insertion status of the camera includes:

[0039] Position the camera's insertion status;

[0040] When the camera is in the plug-in state, locate the interrupt node of the video stream;

[0041] Define the interruption part of the video stream based on the interruption node of the video stream;

[0042] Trigger active repair logic based on interrupted parts of the video stream, and regulate the video stream in the active repair logic;

[0043] The resumption of the video stream is triggered based on the regulation of the video stream.

[0044] Optionally, during the video stream recovery process, monitoring the recovery progress of the video stream and defining the recovery progress of each camera until the recovery of each camera is completed, includes:

[0045] During the video stream recovery process, monitor the video stream recovery progress;

[0046] Determine the recovery progress of each camera based on the recovery progress of the video stream;

[0047] Position the moving direction of the vehicle and the environmental image corresponding to the surrounding environment;

[0048] Defining the priority of detecting the orientation according to the moving direction of the vehicle and the environmental image corresponding to the surrounding environment;

[0049] Match the priority of each camera according to the priority of the detection orientation;

[0050] Matching the restoration priority of each camera based on the priority of each camera;

[0051] The recovery of each camera is triggered in turn according to the recovery priority of each camera, and the recovery progress of each camera is defined in turn until the recovery of each camera is completed.

[0052] According to one aspect of an embodiment of the present application, there is provided an active repair device for each camera in a vehicle, comprising:

[0053] A collection module is used for arranging multiple cameras around the vehicle and synchronously collecting the surrounding environment of the vehicle as the vehicle travels; collecting the camera links of multiple cameras relative to the deserializer;

[0054] An interrupt module, used to define the interrupt part of the camera based on the camera link and the connection status of the camera;

[0055] A detection module, used for dividing the first part and the second part based on the interruption part of the camera, and triggering the hot plug detection logic based on the second part;

[0056] A status module, used to define the insertion status of the camera based on the status of the radio frequency communication link and the video stream corresponding to the camera in the hot plug detection logic;

[0057] A first recovery module, used for triggering recovery of the video stream based on the insertion state of the camera;

[0058] The second recovery module is used to monitor the recovery progress of the video stream during the recovery process of the video stream and define the recovery progress of each camera until the recovery of each camera is completed.

[0059] According to one aspect of an embodiment of the present application, a computer-readable medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the active repair method for each camera in a vehicle as described in the above embodiment is implemented.

[0060] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the active repair method for each camera in a vehicle as described in the above embodiments.

[0061] According to one aspect of an embodiment of the present application, a computer program product or a computer program is provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the active repair method for each camera in a vehicle provided in the above embodiment.

[0062] In the technical solutions provided in some embodiments of the present application, multiple cameras are arranged on the side of the vehicle, and synchronously collect the surrounding environment of the vehicle as the vehicle travels; collect camera links of multiple cameras relative to the deserializer; define the interruption part of the camera based on the camera link and the connection status of the camera, and define a second part for the interruption part of the camera, so as to trigger the hot plug detection logic based on the second part, and define the insertion state of the camera according to the hot plug detection logic, so as to control the insertion state of the camera under the state of the RF communication link and the video stream corresponding to the camera, thereby ensuring the accuracy of the insertion state of the camera, and at the same time, triggering the recovery of the video stream based on the insertion state of the camera; in the process of recovering the video stream, monitor the recovery progress of the video stream, and define the recovery progress of each camera until the recovery of each camera is completed, so as to complete the recovery of the insertion state of the camera and the video stream, thereby ensuring that each camera can achieve self-repair during the dynamic movement of the vehicle, and can respond to the interruption of each camera in time, thereby increasing the robustness of the system and the user experience.

[0063] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0065] Figure 1 A schematic diagram showing a process flow of an active repair method for each camera in a vehicle according to an embodiment of the present application is shown;

[0066] Figure 2 Shows Figure 1 Schematic diagram of the process of S110;

[0067] Figure 3 Shows Figure 1 Schematic diagram of the process of S120;

[0068] Figure 4 Shows Figure 1 Schematic diagram of the process flow of S130;

[0069] Figure 5 Shows Figure 1 Schematic diagram of the process of S140;

[0070] Figure 6 Shows Figure 1 Schematic diagram of the process flow of S150;

[0071] Figure 7 Shows Figure 1 Schematic diagram of the process of S160;

[0072] Figure 8 A block diagram of an active repair device for each camera in a vehicle according to an embodiment of the present application is shown;

[0073] Fig. 9 A schematic diagram of the structure of a computer system suitable for implementing the camera of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0074] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.

[0075] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0076] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0077] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0078] Figure 1 A schematic flow chart of an active repair method for each camera in a vehicle according to an embodiment of the present application is shown. The method can be applied to an active repair scenario of a camera.

[0079] See also Figures 1 to 9 A method for maintaining the status of a portable printer is applied to a scenario of diagnosing the status of a portable printer. The method for maintaining the status of a portable printer comprises:

[0080] Step S110: multiple cameras are arranged around the vehicle, and synchronously collect the surrounding environment of the vehicle as the vehicle travels; and camera links of the multiple cameras relative to the deserializer are collected;

[0081] Step S120: defining an interruption portion of the camera based on the camera link and the connection status of the camera;

[0082] Step S130: dividing the first part and the second part based on the interruption part of the camera, and triggering the hot plug detection logic based on the second part;

[0083] Step S140: in the hot plug detection logic, defining the insertion status of the camera based on the status of the radio frequency communication link and the video stream corresponding to the camera;

[0084] Step S150: triggering the recovery of the video stream based on the insertion status of the camera;

[0085] Step S160: During the recovery process of the video stream, the recovery progress of the video stream is monitored, and the recovery progress of each camera is defined until the recovery of each camera is completed.

[0086] In an embodiment of the present invention, through the method in the embodiment of the present invention, multiple cameras are arranged on the side of the vehicle, and synchronously collect the surrounding environment of the vehicle as the vehicle travels; collect the camera links of multiple cameras relative to the deserializer; define the interruption part of the camera based on the camera link and the connection status of the camera, and define the second part for the interruption part of the camera, so as to trigger the hot plug detection logic based on the second part, and define the insertion state of the camera according to the hot plug detection logic, so as to control the insertion state of the camera under the state of the radio frequency communication link and the video stream corresponding to the camera, and ensure the accuracy of the insertion state of the camera, and at the same time, trigger the recovery of the video stream based on the insertion state of the camera; in the process of video stream recovery, monitor the recovery progress of the video stream, and define the recovery progress of each camera until the recovery of each camera is completed, so as to complete the insertion state of the camera and the recovery of the video stream, ensure that each camera can achieve self-repair with the dynamic movement of the vehicle, and can respond to the interruption of each camera in time, increase the robustness of the system and user experience. In one embodiment, the first part and the second part can be the upper half of the interruption and the lower half of the interruption, respectively.

[0087] refer to Figure 2 In step S110, a plurality of cameras are arranged around the vehicle, and synchronously collect the surrounding environment of the vehicle as the vehicle travels; and collect the camera links of the plurality of cameras relative to the deserializer;

[0088] In the specific implementation process of the present invention, the specific steps may be:

[0089] S111: Collecting a three-dimensional model of the vehicle;

[0090] S112: defining an outer contour of the vehicle based on the three-dimensional model of the vehicle, and defining a detection range according to the outer contour of the vehicle and a moving speed of the vehicle;

[0091] S113: defining installation positions of a plurality of cameras based on the detection range and the outer contour of the vehicle, and arranging corresponding cameras along the installation positions of the plurality of cameras, so that the plurality of cameras are arranged around the vehicle, and the plurality of cameras synchronously collect the surrounding environment of the vehicle as the vehicle travels;

[0092] S114: locating multiple cameras and deserializers, and synchronously controlling the multiple cameras based on the deserializers;

[0093] S115: Matching camera links of the multiple cameras relative to the deserializer based on response paths of the deserializer to the multiple cameras, so as to complete acquisition of the camera links of the multiple cameras relative to the deserializer.

[0094] In an embodiment of the present application, a three-dimensional model of the vehicle is collected; the outer contour of the vehicle is defined based on the three-dimensional model of the vehicle, and a detection range is defined according to the outer contour of the vehicle and the moving speed of the vehicle, so as to dynamically define the detection range, thereby defining the installation positions of multiple cameras based on the detection range and the outer contour of the vehicle, and arranging corresponding cameras along the installation positions of the multiple cameras, so that the multiple cameras are set on the surrounding side of the vehicle, and the multiple cameras synchronously collect the surrounding environment of the vehicle as the vehicle travels.

[0095] At this time, multiple cameras and deserializers are located, and the multiple cameras are synchronously controlled based on the deserializers; the camera links of the multiple cameras relative to the deserializers are matched based on the response paths of the deserializers to the multiple cameras to complete the acquisition of the camera links of the multiple cameras relative to the deserializers, so as to further control the camera links of the multiple cameras relative to the deserializers.

[0096] refer to Figure 3 , in step S120, defining the interruption portion of the camera based on the camera link and the connection state of the camera;

[0097] In the specific implementation process of the present invention, the specific steps may be:

[0098] S121: Get the connection status of the camera;

[0099] S122: Associating the camera link and the connection status of the camera;

[0100] S123: defining an abnormal location based on the camera link and the connection status of the camera;

[0101] S124: Matching a corresponding abnormal path according to the abnormal position;

[0102] S125: Define the interruption part of the camera according to the abnormal path and the connection status of the camera.

[0103] In an embodiment of the present application, the connection status of the camera is obtained, and the camera link and the connection status of the camera are introduced to facilitate associating the camera link and the connection status of the camera, so as to define the abnormal position based on the camera link and the connection status of the camera, thereby clarifying the abnormal position and achieving accurate positioning of the abnormal position.

[0104] At this time, the corresponding abnormal path is matched according to the abnormal position, so as to control the abnormal path, and then the interrupted part of the camera is defined according to the abnormal path and the connection status of the camera, so as to clarify the interrupted part of the camera and achieve further control of the interrupted part of the camera.

[0105] refer to Figure 4, in step S130, dividing the first part and the second part based on the interruption part of the camera, and triggering the hot plug detection logic based on the second part;

[0106] In the specific implementation process of the present invention, the specific steps may be:

[0107] S131: the interruption part of the freeze camera;

[0108] S132: Divide the first part and the second part according to the interruption part of the camera and the interruption node of the camera;

[0109] S133: triggering the execution logic based on the first part, marking a flag bit indicating that the execution is completed in the execution logic;

[0110] S134: triggering the second part based on the response of the flag bit indicating that the execution is completed, and at this time, the second part is adjusted from the sleep state to the working state;

[0111] S135: When the second part is in the working state, the corresponding operation content is executed based on the second part, and the hot plug detection logic is triggered.

[0112] In an embodiment of the present application, the interrupt part of the camera is frozen; the first part and the second part are divided according to the interrupt part of the camera and the interrupt node of the camera; the execution logic is triggered based on the first part, and the execution completion mark bit is marked in the execution logic, thereby introducing the execution completion mark bit to facilitate further response based on the execution completion mark bit.

[0113] Therefore, the second part is triggered based on the response of the execution completion mark bit. At this time, the second part is adjusted from the sleep state to the working state; when the second part is in the working state, the corresponding operation content is executed based on the second part, and the hot plug detection logic is triggered.

[0114] refer to Figure 5 , S140: in the hot plug detection logic, defining the insertion state of the camera based on the state of the radio frequency communication link and the video stream corresponding to the camera;

[0115] In the specific implementation process of the present invention, the specific steps may be:

[0116] S141: In the hot plug detection logic, collecting the radio frequency communication link corresponding to the camera;

[0117] S142: traverse the radio frequency communication link corresponding to the camera, and define the status of the radio frequency communication link;

[0118] S143: defining a first state parameter based on the state of the radio frequency communication link;

[0119] S144: collecting video streams; defining interruption nodes of the video streams based on traversal of the video streams;

[0120] S145: defining the interruption state of the video stream according to the interruption node of the video stream, and defining a second state parameter; associating the second state parameter with the first state parameter, and defining the insertion state of the camera based on the second state parameter and the first state parameter.

[0121] In an embodiment of the present application, multiple cameras are arranged on the surrounding side of the vehicle, and synchronously collect the surrounding environment of the vehicle as the vehicle travels; the camera links of multiple cameras relative to the deserializer are collected; the interruption part of the camera is defined based on the camera link and the connection status of the camera, and a second part is defined for the interruption part of the camera, so as to trigger the hot plug detection logic based on the second part, and the insertion status of the camera is defined according to the hot plug detection logic, so as to control the insertion status of the camera under the status of the RF communication link and the video stream corresponding to the camera, thereby ensuring the accuracy of the insertion status of the camera.

[0122] In the hot plug detection logic, the RF communication link corresponding to the camera is collected and managed. At the same time, the RF communication link corresponding to the camera is traversed and the state of the RF communication link is defined, so as to define the first state parameter based on the state of the RF communication link.

[0123] At this time, the video stream is collected; based on the traversal of the video stream, the interruption node of the video stream is defined; according to the interruption node of the video stream, the interruption state of the video stream is defined, and the second state parameter is defined, so as to introduce the second state parameter and the first state parameter, so as to control the second state parameter and the first state parameter as a whole. Further, the second state parameter and the first state parameter are associated, and the insertion state of the camera is defined based on the second state parameter and the first state parameter.

[0124] refer to Figure 6 , S150: triggering the recovery of the video stream based on the insertion status of the camera;

[0125] In the specific implementation process of the present invention, the specific steps may be:

[0126] S151: Positioning the insertion state of the camera;

[0127] S152: when the camera is in the inserted state, locating the interruption node of the video stream;

[0128] S153: defining an interruption portion of the video stream based on an interruption node of the video stream;

[0129] S154: triggering active repair logic according to the interrupted part of the video stream, and regulating the video stream in the active repair logic; triggering recovery of the video stream based on the regulation of the video stream.

[0130] In an embodiment of the present application, the insertion state of the camera is located; when the camera is in the insertion state, the interruption node of the video stream is located; based on the interruption node of the video stream, the interruption part of the video stream is defined to facilitate the positioning of the interruption part of the video stream and further control the interruption part of the video stream.

[0131] Furthermore, active repair logic is triggered according to the interrupted part of the video stream, and the video stream is regulated by the active repair logic; based on the regulation of the video stream, the recovery of the video stream is triggered, thereby realizing the recovery of the video stream and ensuring real-time control of the video stream.

[0132] refer to Figure 7 , S160: During the video stream recovery process, monitor the recovery progress of the video stream and define the recovery progress of each camera until the recovery of each camera is completed;

[0133] In the specific implementation process of the present invention, the specific steps may be:

[0134] S161: During the video stream recovery process, monitoring the video stream recovery progress;

[0135] S162: Determine the recovery progress of each camera based on the recovery progress of the video stream;

[0136] S163: Locate the moving direction of the vehicle and the environment image corresponding to the surrounding environment;

[0137] S164: defining the priority of detecting the orientation according to the moving direction of the vehicle and the environment image corresponding to the surrounding environment;

[0138] S165: matching the priority of each camera according to the priority of the detection orientation;

[0139] S166: Matching the restoration priority of each camera based on the priority of each camera;

[0140] S167: Triggering the recovery of each camera in turn according to the recovery priority of each camera, and defining the recovery progress of each camera in turn until the recovery of each camera is completed.

[0141] In the specific implementation process of the present invention, the recovery of the video stream is triggered based on the insertion status of the camera; during the recovery process of the video stream, the recovery progress of the video stream is monitored, and the recovery progress of each camera is defined until the recovery of each camera is completed, so as to complete the insertion status of the camera and the recovery of the video stream, ensuring that each camera can achieve self-repair during the dynamic movement of the vehicle, and can respond to the interruption of each camera in time, thereby increasing the robustness of the system and user experience.

[0142] At this time, during the recovery process of the video stream, the recovery progress of the video stream is monitored; the recovery progress of each camera is determined based on the recovery progress of the video stream, and at the same time, the moving direction of the vehicle and the environmental image corresponding to the surrounding environment are located, so as to define the priority of the detection direction according to the moving direction of the vehicle and the environmental image corresponding to the surrounding environment, thereby controlling the priority of the detection direction.

[0143] Furthermore, the priority of each camera is matched according to the priority of the detection orientation; the recovery priority of each camera is matched based on the priority of each camera; the recovery of each camera is triggered in sequence according to the recovery priority of each camera, and the recovery progress of each camera is defined in sequence until the recovery of each camera is completed.

[0144] The details are as follows:

[0145] The solution of the present invention can be applied to smart terminals such as domain controllers or smart cockpits, including: a CPU, a deserializer, multiple serializers and sensors, wherein the sensor and the serializer form a camera, the four-way module is connected to the deserializer through a coaxial cable, the deserializer aggregates the video of the four-way camera and transmits it to the CPU through the MIPI interface, and the CPU can transmit some control instructions and query status information through the I2C interface.

[0146] The lockpin pin is a pin on the deserializer, which is connected to the CPU. By configuring the register of the deserializer, the status of the camera coaxial line link connection can be mapped to this pin. If an abnormality occurs, a falling edge interrupt will be triggered to notify the CPU for processing.

[0147] The method of the present invention can be applied in operating systems such as Linux, Android, and QNX. By configuring the lock pin, a GPIO pin on the deserializer is connected to the CPU. By configuring the register of the deserializer, the state of the camera coaxial line link connection can be mapped to the pin. If an abnormality occurs, a falling edge interrupt will be triggered to notify the CPU to process. Based on the interrupt trigger signal of the deserializer link disconnection, after receiving the interruption, I2C (synchronous serial communication bus, generally CPU as the master, peripheral chip as the slave, used for CPU and peripherals to communicate data, in this case used to read and write registers of deserializer serializer and sensor, configure and query status information) is used to detect which camera link is disconnected, and then the serializer and sensor are reconfigured for recovery after the link is normal. After recovery, the MIPI video data can be normally transmitted to the SOC.

[0148] Step S1, initialization process, handles the scenario where the camera is missing when powered on and is plugged in again after power on, that is, the camera is not fully connected when powered on, and is plugged in again after power on. If it is not handled, the initialization process is missed, and the camera can only be initialized when it is powered on and reset next time. It can be achieved that as long as the camera is connected anytime and anywhere, a video stream will be generated immediately, with good real-time performance and fast recovery, without the need for cold start reset and other operations, and the camera can be hot-swapped at any time.

[0149] After initialization and power-on, the software first reads the configuration file to confirm how many cameras are connected to the deserializer, and then reads the link lock status to determine whether all cameras are linked. If all cameras are connected, it means that all camera links are normal when powered on. After configuring the serial deserializer and sensor, the SOC can receive the video stream data sent by MIPI; if there are cameras that are not connected, it means that the cameras are not fully connected when powered on, but no interrupt will be triggered at this time, so you need to manually create a second part of the thread to handle the hot plug detection logic.

[0150] S2, then continue to configure the remaining deserializers and sensors to ensure that the remaining cameras can output video streams normally, and finally register the lock pin interrupt to wait for the hot plug event to occur. If it occurs, enter S2.

[0151] Step S2, the second part (interrupt processing flow, in order to ensure the real-time nature of the interrupt, the interrupt processing is generally divided into the upper half and the second part,

[0152] When a hardware interrupt occurs, the first part is entered first. The upper part needs to be executed very quickly and cannot execute code that may cause blocking. The upper part usually sets a flag bit and sends a signal to wake up the second part, which will exit immediately.

[0153] In the second part, some time-consuming operations are processed. The second part is generally a thread, which is normally dormant, waiting for the wake-up signal sent by the first part to wake up and process the interrupt.

[0154] Hot plug detection logic. When the camera link is suddenly disconnected under normal circumstances, the system will enter the second part to execute the hot plug detection logic. First, read the registers of the link lock and video lock status through I2C, and then enter S3.

[0155] Step S3, check the link lock (link lock indicates whether the deserializer and serializer have a normal link through the coaxial line (there is a handshake signal on the hardware, the serializer and deserializer frequencies are configured to be the same in the software, and the hardware will automatically handshake to establish a successful link. The status of each coaxial line link has a bit corresponding to a register in the deserializer. Reading the register of the deserializer can confirm the connection status of all cameras) status, confirm the specific status of multiple cameras connected to the deserializer, and make different processing. If link lock is not true, set the camera to the unplugged state, jump to S2 to continue reading the status until link lock is true (as mentioned earlier, the explanation of link lock is that this is a hardware behavior. After the camera is connected, the deserializer and serializer will handshake successfully to establish a link (RF communication link) through the coaxial line. This is to wait for the hardware link connection to be completed); if link lock is true, it is necessary to enter S4 to check the status of video lock.

[0156] Step S4, check the video lock status (video lock is a sign of camera video lock. After configuring the sensor register and opening the MIPI output, the sensor data will enter the serializer and be transmitted to the deserializer through the coaxial cable after encoding. After decoding, the deserializer can recognize the complete video frame and set the bit of the video lock register, indicating that the deserializer has received the correct video frame sent by the sensor. It is usually used to detect whether the sensor and the serializer are configured successfully) to confirm the specific status of multiple cameras connected to the deserializer and make different processing.

[0157] If video lock is not true, it means the camera has been plugged in, but the video stream has not been restored. Enter S5 to restore the camera video stream. If video lock is true, it means the camera has been plugged in and the video stream output is normal. Set the camera status to the plugged in status and enter S6 for the final double check.

[0158] Step S5, video stream recovery. Since the coaxial line transmits data and provides power, once it is disconnected and then restored, the serializer and sensor of the camera will be reset to the initial state and need to be reconfigured to output the video stream.

[0159] First, disable all I2C control channels, open the I2C control channel to be restored, and ensure that there is only one I2C communication at this moment. This can avoid address conflicts caused by the default I2C addresses of all modules. Then use the default I2C address to reset the I2C address of the serializer and sensor in this module, and then open all I2C controls, use a new set of addresses to operate the serializer and sensor, and then load the configuration information of frame synchronization and sensor setting (the sensor manufacturer will give the configuration sequence, that is, the corresponding register setting corresponding value) to the corresponding registers of the serializer and sensor through the I2C bus. After execution, enter S3. If the recovery is successful, it will smoothly enter S4 to exit. If the connection is disconnected again during the recovery process, it will be detected that the recovery is not good, and this step S5 will be repeated.

[0160] Therefore, the present invention uses the lock pin to quickly identify a camera link anomaly in an interrupt manner, and then further confirms the specific camera link status by reading the status register through I2C. After the hardware link is restored, the video data stream can be quickly restored without continuously powering on the intelligent terminal or resetting the software, thereby achieving hot-swappable cameras.

[0161] In an embodiment of the present invention, through the method in the embodiment of the present invention, multiple cameras are set on the side of the vehicle, and synchronously collect the surrounding environment of the vehicle as the vehicle travels; collect camera links of multiple cameras relative to the deserializer; define the interruption part of the camera based on the camera link and the connection status of the camera, and define the second part for the interruption part of the camera, so as to trigger the hot plug detection logic based on the second part, and define the insertion state of the camera according to the hot plug detection logic, so as to control the insertion state of the camera under the state of the radio frequency communication link and the video stream corresponding to the camera, thereby ensuring the accuracy of the insertion state of the camera, and at the same time, triggering the recovery of the video stream based on the insertion state of the camera; in the process of video stream recovery, monitor the recovery progress of the video stream, and define the recovery progress of each camera until the recovery of each camera is completed, so as to complete the recovery of the insertion state of the camera and the video stream, thereby ensuring that each camera can achieve self-repair during the dynamic movement of the vehicle, and can respond to the interruption of each camera in time, thereby increasing the robustness of the system and the user experience.

[0162] The following describes an embodiment of the device of the present application, which can be used to perform the active repair method of each camera in the vehicle in the above embodiment of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the active repair method of each camera in the vehicle in the above embodiment of the present application.

[0163] Figure 8A block diagram of an active repair device for each camera in a vehicle according to an embodiment of the present application is shown.

[0164] Reference Figure 8 As shown, according to an embodiment of the present application, an active repair device for each camera in a vehicle includes:

[0165] The acquisition module 210 is used to set multiple cameras around the vehicle and synchronously acquire the surrounding environment of the vehicle as the vehicle travels; acquire the camera links of the multiple cameras relative to the deserializer;

[0166] An interrupt module 220 for defining an interrupt portion of the camera based on the camera link and the connection status of the camera;

[0167] A detection module 230, configured to divide the camera into a first part and a second part based on the interruption part, and trigger a hot plug detection logic based on the second part;

[0168] A state module 240, used to define the insertion state of the camera based on the state of the radio frequency communication link and the video stream corresponding to the camera in the hot plug detection logic;

[0169] A first recovery module 250, configured to trigger recovery of a video stream based on the insertion status of a camera;

[0170] The second recovery module 260 is used to monitor the recovery progress of the video stream during the recovery process of the video stream, and define the recovery progress of each camera until the recovery of each camera is completed.

[0171] Fig. 9 A schematic diagram of the structure of a computer system suitable for implementing the camera of an embodiment of the present application is shown.

[0172] It should be noted that Fig. 9 The computer system of the camera shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0173] like Fig. 9As shown, the computer system includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 302 or the program loaded from the storage part 308 to the random access memory (RAM) 303, such as executing the method described in the above embodiment. In RAM 303, various programs and data required for system operation are also stored. CPU 301, ROM 302 and RAM 303 are connected to each other through bus 304. Input / output (I / O) interface 305 is also connected to bus 304.

[0174] The following components are connected to the I / O interface 305: an input section 306 including a keyboard, a mouse, etc.; an output section 307 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 305 as needed. A removable medium 311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 310 as needed so that a computer program read therefrom is installed into the storage section 308 as needed.

[0175] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 309, and / or installed from a removable medium 311. When the computer program is executed by a central processing unit (CPU) 301, various functions defined in the system of the present application are executed.

[0176] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0177] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0178] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0179] As another aspect, the present application further provides a computer-readable medium, which may be included in the camera described in the above embodiment; or may exist independently without being assembled into the camera. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a camera, the camera implements the method described in the above embodiment.

[0180] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0181] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0182] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

[0183] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for actively repairing each camera in a vehicle, characterized in that: include: Multiple cameras are arranged around the vehicle and synchronously collect the surrounding environment of the vehicle as the vehicle travels; Capture camera links of multiple cameras relative to deserializers; Defining an interrupt portion of the camera based on the camera link and the connection status of the camera; Divide the first part and the second part based on the interrupt part of the camera, and trigger the hot plug detection logic based on the second part; In the hot-plug detection logic, the camera insertion status is defined based on the status of the RF communication link and video stream corresponding to the camera; Triggering the resumption of video streaming based on the camera's insertion status; During the recovery process of the video stream, monitor the recovery progress of the video stream and define the recovery progress of each camera until the recovery of each camera is completed.

2. The method according to claim 1, characterized in that The multiple cameras are arranged around the vehicle and synchronously collect the surrounding environment of the vehicle as the vehicle travels; collecting the camera links of the multiple cameras relative to the deserializer includes: Collect a three-dimensional model of the vehicle; An outer contour of the vehicle is defined based on a three-dimensional model of the vehicle, and a detection range is defined according to the outer contour of the vehicle and a moving speed of the vehicle; Defining installation positions of multiple cameras based on the detection range and the outer contour of the vehicle, and arranging corresponding cameras along the installation positions of the multiple cameras, so that the multiple cameras are arranged around the vehicle, and the multiple cameras synchronously collect the surrounding environment of the vehicle as the vehicle travels; Locate multiple cameras and deserializers, and synchronously control multiple cameras based on the deserializers; Camera links of the multiple cameras relative to the deserializer are matched based on response paths of the deserializer to the multiple cameras, so as to complete the acquisition of the camera links of the multiple cameras relative to the deserializer.

3. The method according to claim 1, characterized in that The interruption part of defining the camera based on the camera link and the connection state of the camera includes: Get the camera's connection status; Associate the camera link and the camera's connection status; Defining abnormal locations based on camera links and camera connection status; Match the corresponding abnormal path according to the abnormal location; Define the interruption section of the camera based on the abnormal path and the connection status of the camera.

4. The method according to claim 3, characterized in that The camera-based interruption part is divided into a first part and a second part, and a hot plug detection logic is triggered based on the second part, including: freeze-frame camera interruptions; Dividing the first part and the second part according to the interruption part of the camera and the interruption node of the camera; Based on the first part, the execution logic is triggered, and a flag bit indicating that the execution is completed is marked in the execution logic; The second part is triggered based on the response of the execution completion flag bit, and at this time, the second part is adjusted from the sleep state to the working state; When the second part is in the working state, the corresponding operation content is executed based on the second part, and the hot plug detection logic is triggered.

5. The method according to claim 4, characterized in that In the hot plug detection logic, the insertion state of the camera is defined based on the state of the radio frequency communication link and the video stream corresponding to the camera, including: In the hot-plug detection logic, the RF communication link corresponding to the camera is collected; Traverse the radio frequency communication link corresponding to the camera and define the status of the radio frequency communication link; defining a first state parameter based on a state of the radio frequency communication link; Collect video streams; define interrupt nodes of the video streams based on the traversal of the video streams; The interruption state of the video stream is defined according to the interruption node of the video stream, and a second state parameter is defined; the second state parameter and the first state parameter are associated, and the insertion state of the camera is defined based on the second state parameter and the first state parameter.

6. The method according to claim 5, characterized in that The recovery of the video stream is triggered based on the insertion state of the camera, including: Position the camera's insertion status; When the camera is in the plug-in state, locate the interrupt node of the video stream; Define the interruption part of the video stream based on the interruption node of the video stream; The active repair logic is triggered according to the interrupted part of the video stream, and the video stream is regulated by the active repair logic; and the recovery of the video stream is triggered based on the regulation of the video stream.

7. The method according to claim 6, characterized in that During the video stream recovery process, the recovery progress of the video stream is monitored, and the recovery progress of each camera is defined until the recovery of each camera is completed, including: During the video stream recovery process, monitor the video stream recovery progress; Determine the recovery progress of each camera based on the recovery progress of the video stream; Position the moving direction of the vehicle and the environmental image corresponding to the surrounding environment; Defining the priority of detecting the orientation according to the moving direction of the vehicle and the environmental image corresponding to the surrounding environment; Match the priority of each camera according to the priority of the detection orientation; Matching the restoration priority of each camera based on the priority of each camera; The recovery of each camera is triggered in turn according to the recovery priority of each camera, and the recovery progress of each camera is defined in turn until the recovery of each camera is completed.

8. An active repair device for each camera in a vehicle, characterized in that: include: A collection module is used for arranging multiple cameras around the vehicle and synchronously collecting the surrounding environment of the vehicle as the vehicle travels; Capture camera links of multiple cameras relative to deserializers; An interrupt module, used to define the interrupt part of the camera based on the camera link and the connection status of the camera; A detection module, used for dividing the first part and the second part based on the interruption part of the camera, and triggering the hot plug detection logic based on the second part; A status module, used to define the insertion status of the camera based on the status of the radio frequency communication link and the video stream corresponding to the camera in the hot plug detection logic; A first recovery module, used for triggering recovery of the video stream based on the insertion state of the camera; The second recovery module is used to monitor the recovery progress of the video stream during the recovery process of the video stream and define the recovery progress of each camera until the recovery of each camera is completed.

9. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the active repair method for each camera in a vehicle according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the active repair method for each camera in a vehicle as described in any one of claims 1 to 7.

Images