Vehicle-mounted camera state data returning method and device, vehicle and storage medium

Abstract

The invention relates to the technical field of intelligent driving, in particular to a vehicle-mounted camera state data returning method and device, a vehicle and a storage medium, and the method comprises the steps: determining the target resolution of a current output image based on the obtained current operation state data of a vehicle-mounted camera and the resolution of the current output image, the method comprises the following steps: acquiring current running state data of a vehicle, embedding the current running state data into a current output image based on a target resolution to obtain a target output image, transmitting the target output image back to image receiving equipment of the vehicle, and analyzing the target output image through the image receiving equipment to obtain the current running state data and the current output image. Therefore, the problems that communication efficiency is low, external interference is prone to occurring, system resources are occupied, and the requirement for real-time monitoring is difficult to meet due to the fact that communication interfaces such as I2C have access to the returned image are solved, the state data are embedded into the last line of the image, and the image transmission channel is reused, so that data returning efficiency and system stability are improved.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, and particularly relates to a method, device, vehicle, and storage medium for transmitting in-vehicle camera status data. Background Art

[0002] With the continuous development of vehicle intelligence, in-vehicle cameras are increasingly widely used. As an important device for environmental perception in an intelligent driving system, in-vehicle cameras need to monitor the internal status of the cameras in real time (such as temperature, frame rate, fault codes, etc.) to avoid camera failures and ensure the stable operation of the vehicle.

[0003] In related technologies, the status monitoring of in-vehicle cameras usually accesses the status of in-vehicle cameras periodically through communication interfaces such as I2C.

[0004] However, the access methods of communication interfaces such as I2C (Inter-Integrated Circuit) are difficult to meet the requirements of real-time monitoring due to problems such as low communication efficiency, susceptibility to external interference, and occupation of system resources, and urgent solutions are needed. Summary of the Invention

[0005] This application provides a method, device, vehicle, and storage medium for transmitting in-vehicle camera status data to solve problems such as low communication efficiency, susceptibility to external interference, occupation of system resources, and difficulty in meeting the requirements of real-time monitoring caused by accessing and transmitting images through communication interfaces such as I2C.

[0006] The first aspect embodiment of this application provides a method for transmitting in-vehicle camera status data, including the following steps:

[0007] Obtain the current operating status data of the in-vehicle camera and the resolution of the current output image;

[0008] Determine the target resolution of the current output image according to the current operating status data and the resolution of the current output image, and embed the current operating status data into the current output image based on the target resolution to obtain a target output image;

[0009] Transmit the target output image to the image receiving device of the vehicle, and parse the target output image through the image receiving device to obtain the current operating status data and the current output image.

[0010] Through the above technical solution, by designing the current output image to have multiple outputs in one row, that is, adding a new row at the end of the horizontal resolution of the current output image, and then embedding the current operating status data of the vehicle-mounted camera into the last row of the image, it is possible to directly transmit the current operating status data of the vehicle-mounted camera and the current output image to the image receiving device together, thus avoiding the need to set up I2C communication additionally, and greatly improving the data transmission efficiency and reliability.

[0011] Optionally, in an embodiment of the present application, the obtaining the current operating status data of the vehicle-mounted camera includes:

[0012] Obtaining the dynamic operating status data and static performance data of the vehicle-mounted camera;

[0013] Obtaining the current operating status data according to the dynamic operating status data and the static performance data;

[0014] Wherein, the dynamic operating status data includes the real-time temperature data of the vehicle-mounted camera, the real-time frame rate of the current output image, the real-time total number of frames of the current output image, and the fault code of the vehicle-mounted camera, and the static performance data includes the code and firmware version number of the vehicle-mounted camera.

[0015] Through the above technical solution, based on the obtained current operating status data of the vehicle-mounted camera and the resolution of the current output image, it is possible to detect the operating status of the vehicle-mounted camera and make timely adjustments in case of a fault, thereby improving the safety of vehicle driving.

[0016] Optionally, in an embodiment of the present application, the embedding the current operating status data into the current output image based on the target resolution to obtain a target output image includes:

[0017] Determining the header mark, tail mark and check code of the current operating status data;

[0018] Encapsulating the current operating status data based on the operating status data, the header mark, the tail mark and the check code to obtain an operating status data packet;

[0019] Based on the target resolution, determining the embedding position of the operating status data packet, and embedding the operating status data packet into the current output image correspondingly based on the embedding position to obtain the target output image.

[0020] Through the above technical solution, by automatically embedding the status data into the current output image and replacing the piecemeal reading of I2C, it is possible to achieve strict synchronization between the status of the vehicle-mounted camera and the current output image frame, thus ensuring the data transmission efficiency and system stability.

[0021] Optionally, in an embodiment of the present application, after parsing the target output image by the image receiving device to obtain the current operating state data and the current output image, the following is further included:

[0022] Based on the current operating state data, determine whether the in-vehicle camera meets a preset fault handling condition;

[0023] If the in-vehicle camera meets the preset fault handling condition, determine a fault handling strategy based on the current operating state data, and perform fault handling according to the fault handling strategy.

[0024] Through the above technical solution, by real-time monitoring the operating state of the in-vehicle camera (such as temperature, frame rate, resolution, etc.), potential problems or abnormal situations can be quickly identified. Once a problem is found, the corresponding fault handling strategy is immediately activated, reducing unexpected downtime, extending the service life of the device, improving the reliability and security of the system, and at the same time avoiding problems such as image distortion and delay caused by hardware failures, thereby providing a smoother and more reliable user experience.

[0025] Optionally, in an embodiment of the present application, before parsing the target output image by the image receiving device to obtain the current operating state data and the current output image, the following is further included:

[0026] Determine whether to receive the verification failure information of the target output image of the image receiving device;

[0027] If the verification failure information is received, retransmit the target output image based on the verification failure information.

[0028] Through the above technical solution, when the verification fails, sending the verification failure information of the target output image to the in-vehicle camera helps the in-vehicle camera to retransmit the target output image to the image receiving device in a timely manner based on the verification failure information when receiving the verification failure information, thereby ensuring that the data can finally reach the image receiving device correctly and without error, thus improving the reliability of the entire system.

[0029] According to the method for transmitting vehicle-mounted camera status data according to an embodiment of the present application, the target resolution of the current output image is determined based on the obtained current operating status data of the vehicle-mounted camera and the resolution of the current output image, and the current operating status data is embedded into the current output image based on the target resolution to obtain a target output image. The target output image is transmitted back to the image receiving device of the vehicle, and the current operating status data and the current output image are obtained by parsing the target output image through the image receiving device. Thus, the problems caused by accessing and transmitting back images through communication interfaces such as I2C, such as low communication efficiency, susceptibility to external interference, occupation of system resources, and difficulty in meeting the requirements of real-time monitoring, are solved. By embedding the status data at the end of the image and reusing the image transmission channel, the data transmission efficiency and system stability are improved.

[0030] The second aspect of the present application provides an apparatus for transmitting vehicle-mounted camera status data, including:

[0031] An acquisition module, configured to acquire the current operating status data of the vehicle-mounted camera and the resolution of the current output image;

[0032] A data embedding module, configured to determine the target resolution of the current output image according to the current operating status data and the resolution of the current output image, and embed the current operating status data into the current output image based on the target resolution to obtain a target output image;

[0033] An analysis module, configured to transmit the target output image back to the image receiving device of the vehicle, and analyze the target output image through the image receiving device to obtain the current operating status data and the current output image.

[0034] Optionally, in an embodiment of the present application, the acquisition module includes:

[0035] A first acquisition unit, configured to acquire the dynamic operating status data and static performance data of the vehicle-mounted camera;

[0036] A second acquisition unit, configured to obtain the current operating status data according to the dynamic operating status data and the static performance data;

[0037] Wherein, the dynamic operating status data includes the real-time temperature data of the vehicle-mounted camera, the real-time frame rate of the current output image, the total number of real-time frames of the current output image, and the fault code of the vehicle-mounted camera, and the static performance data includes the code and firmware version number of the vehicle-mounted camera.

[0038] Optionally, in an embodiment of the present application, the data embedding module includes:

[0039] A determination unit, configured to determine a packet header mark, a packet tail mark, and a check code of the current operating status data;

[0040] An encapsulation unit, configured to encapsulate the current operating status data based on the operating status data, the packet header mark, the packet tail mark, and the check code to obtain an operating status data packet;

[0041] A data embedding unit, configured to determine an embedding position of the operating status data packet based on the target resolution, and embed the operating status data packet into the current output image correspondingly based on the embedding position to obtain the target output image.

[0042] Optionally, in an embodiment of the present application, after the parsing module parses the target output image through the image receiving device to obtain the current operating status data and the current output image, the parsing module further includes:

[0043] A first judgment unit, configured to judge whether the vehicle-mounted camera meets a preset fault handling condition based on the current operating status data;

[0044] A fault handling unit, configured to, if the vehicle-mounted camera meets the preset fault handling condition, determine a fault handling strategy based on the current operating status data, and perform fault handling according to the fault handling strategy.

[0045] Optionally, in an embodiment of the present application, before the parsing module parses the target output image through the image receiving device to obtain the current operating status data and the current output image, the parsing module further includes:

[0046] A second judgment unit, configured to judge whether to receive a check failure message of the target output image of the image receiving device;

[0047] A transmission unit, configured to, if the check failure message is received, retransmit the target output image based on the check failure message.

[0048] The device for transmitting vehicle-mounted camera status data according to an embodiment of the present application determines the target resolution of the current output image based on the acquired current operating status data of the vehicle-mounted camera and the resolution of the current output image, embeds the current operating status data into the current output image based on the target resolution to obtain a target output image, transmits the target output image to the image receiving device of the vehicle, and parses the target output image through the image receiving device to obtain the current operating status data and the current output image. Thus, the problems caused by accessing and transmitting images through communication interfaces such as I2C, such as low communication efficiency, susceptibility to external interference, occupation of system resources, and difficulty in meeting the requirements of real-time monitoring, are solved. By embedding the status data at the end of the image and reusing the image transmission channel, the data transmission efficiency and system stability are improved.

[0049] An embodiment of the third aspect of the present application provides a vehicle, including: a memory, a processor, and a computer program stored on the memory and executable on the processor. The processor executes the program to implement the method for transmitting vehicle-mounted camera status data as described in the above embodiment.

[0050] An embodiment of the fourth aspect of the present application provides a computer-readable storage medium storing computer instructions for causing the computer to execute the method for transmitting vehicle-mounted camera status data as described in the above embodiment.

[0051] An embodiment of the fifth aspect of the present application provides a computer program product including a computer program, which is executed to implement the method for transmitting vehicle-mounted camera status data as described in the above embodiment.

[0052] Additional aspects and advantages of the present application will be given in part in the following description, become apparent in part from the following description, or be understood through the practice of the present application. Description of the Drawings

[0053] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description of the embodiments in conjunction with the drawings, where:

[0054] Figure 1 It is a schematic diagram of a method for transmitting internal operating status data of a vehicle-mounted camera in the related art;

[0055] Figure 2 It is a flowchart of a method for transmitting vehicle-mounted camera status data according to an embodiment of the present application;

[0056] Figure 3 It is a flowchart of a method for transmitting vehicle-mounted camera status data according to an embodiment of the present application;

[0057] Figure 4Schematic diagram of image embedding according to an embodiment of the present application;

[0058] Figure 5 Example diagram of a device for transmitting back the status data of an in-vehicle camera according to an embodiment of the present application;

[0059] Figure 6 Schematic diagram of the structure of a vehicle according to an embodiment of the present application.

[0060] Among them, 10 - device for transmitting back the status data of the in-vehicle camera, 100 - acquisition module, 200 - encapsulation module, 300 - parsing module. Detailed implementation manners

[0061] The embodiments of the present application will be described in detail below. The examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements with the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain the present application, and should not be construed as a limitation to the present application.

[0062] The method, device, vehicle, and storage medium for transmitting back the status data of an in-vehicle camera according to the embodiments of the present application will be described below with reference to the accompanying drawings. In view of the problems of low communication efficiency, susceptibility to external interference, occupation of system resources, and difficulty in meeting the requirements of real-time monitoring caused by accessing and transmitting back images through communication interfaces such as I2C in the above-mentioned background art, the present application provides a method for transmitting back the status data of an in-vehicle camera. In this method, based on the currently obtained operating status data of the in-vehicle camera and the resolution of the currently output image, the target resolution of the currently output image is determined, and the currently operating status data is embedded into the currently output image based on the target resolution to obtain a target output image. The target output image is transmitted back to the image receiving device of the vehicle, and the currently operating status data and the currently output image are obtained by parsing the target output image through the image receiving device. Thereby, the problems of low communication efficiency, susceptibility to external interference, occupation of system resources, and difficulty in meeting the requirements of real-time monitoring caused by accessing and transmitting back images through communication interfaces such as I2C are solved. By embedding the status data at the end of the image and reusing the image transmission channel, the data transmission efficiency and system stability are improved.

[0063] As an important part of intelligent driving technology, in actual applications, based on the requirements of functional safety and information security, it is necessary to monitor the internal operating status of the camera in real time to avoid camera failures and ensure the stable operation of the vehicle system.

[0064] In the related art, such as Figure 1As shown, most of them use communication interfaces such as I2C to periodically access the status of in-vehicle cameras. For example, they access operating parameters such as the temperature and frame rate of in-vehicle cameras. However, due to the inherent rate limitations of the I2C protocol, such as inherent defects like protocol redundancy, serial transmission, and polling mechanism, it is difficult to meet the efficient feedback requirements of in-vehicle camera status. In addition, there are usually multiple devices connected to the I2C protocol bus, and any abnormality in any path may affect the failure of I2C communication. Therefore, for the monitoring of the internal operating status of in-vehicle cameras, in the embodiments of the present application, the internal operating status of the camera is fed back as an image by embedding it into the image row, avoiding the need for additional I2C communication and the need to obtain information one by one in the related art, and greatly improving the data feedback efficiency and reliability.

[0065] Specifically, Figure 2 is a schematic flowchart of a method for feedback of in-vehicle camera status data provided by an embodiment of the present application.

[0066] As Figure 2 shown, the method for feedback of in-vehicle camera status data includes the following steps:

[0067] Step S201, obtain the current operating status data of the in-vehicle camera and the resolution of the current output image.

[0068] Step S202, determine the target resolution of the current output image according to the current operating status data and the resolution of the current output image, and embed the current operating status data into the current output image based on the target resolution to obtain a target output image.

[0069] Step S203, transmit the target output image back to the image receiving device of the vehicle, and parse the target output image through the image receiving device to obtain the current operating status data and the current output image.

[0070] Thus, in the embodiments of the present application, by designing the current output image to have one more output row, that is, adding a new row at the end of the horizontal resolution of the current output image, and then embedding the current operating status data of the in-vehicle camera into the last row of the image, the current operating status data of the in-vehicle camera and the current output image can be directly transmitted back to the image receiving device together, thus avoiding the need to set up additional I2C communication and greatly improving the data feedback efficiency and reliability.

[0071] Among them, step S201 includes the following aspects:

[0072] Optionally, in an embodiment of the present application, obtaining the current operating state data of the vehicle-mounted camera includes: obtaining the dynamic operating state data and static performance data of the vehicle-mounted camera; obtaining the current operating state data based on the dynamic operating state data and the static performance data; wherein, the dynamic operating state data includes the real-time temperature data of the vehicle-mounted camera, the real-time frame rate of the current output image, the real-time total number of frames of the current output image, and the fault code of the vehicle-mounted camera, and the static performance data includes the code and firmware version number of the vehicle-mounted camera.

[0073] Wherein, the resolution of the current output image is the content of the original image, that is, the image content excluding the current operating state data of the vehicle-mounted camera. For example, if the resolution of the current output image is 1280×720, then 1280×720 is the effective picture of the current output image.

[0074] Specifically, in an embodiment of the present application, the current operating state data of the vehicle-mounted camera may include dynamic operating state data and static performance data, wherein the dynamic operating state data includes the real-time temperature data of the vehicle-mounted camera, the real-time frame rate of the current output image, the real-time total number of frames of the current output image, and the fault code of the vehicle-mounted camera, and the static performance data includes the code and firmware version number of the vehicle-mounted camera.

[0075] Specifically, after the vehicle-mounted camera is powered on, it is necessary to perform periodic detection based on the self-operating state of the vehicle-mounted camera and collect the current operating state data in real time. This operating state data can reflect the current health state of the vehicle-mounted camera. For example, the real-time temperature data of the vehicle-mounted camera can be obtained through the internal temperature sensor of the vehicle-mounted camera. When monitoring the acquisition frequency of the current output image, the real-time frame rate of the current output image can be obtained through the frame rate counter, and the real-time total number of frames of the current output image can be obtained by using the frame counter for the acquisition of each frame of the image. In addition to the above real-time state of the vehicle-mounted camera that needs to be accessed periodically, it is also necessary to obtain the static performance data of the vehicle-mounted camera after the vehicle-mounted camera is powered on, such as obtaining the fault code of the vehicle-mounted camera, the manufacturer code and firmware version number of the vehicle-mounted camera, etc.

[0076] For example, the real-time temperature data obtained by the temperature sensor for the vehicle-mounted camera is 65°C, the real-time frame rate of the current output image is 30 frames per second, the fault code of the vehicle-mounted camera is 0x00 (no fault), and the firmware version number is V1.2.3. Thus, subsequent data transmission is performed based on the obtained operating state data above.

[0077] Therefore, based on the current operating state data of the vehicle-mounted camera and the resolution of the current output image obtained above, it is possible to detect the operating state of the vehicle-mounted camera and make timely adjustments in case of a fault, thereby improving vehicle driving safety.

[0078] Step S202 includes the following aspects:

[0079] Optionally, in an embodiment of the present application, encapsulating the current operating state data into the current output image based on the target resolution to obtain a target output image includes: determining the packet header flag, packet tail flag, and check code of the current operating state data; encapsulating the current operating state data based on the operating state data, packet header flag, packet tail flag, and check code to obtain an operating state data packet; determining the embedding position of the operating state data packet based on the target resolution, and embedding the operating state data packet correspondingly into the current output image based on the embedding position to obtain a target output image.

[0080] Specifically, based on the currently obtained operating state data of the vehicle-mounted camera and the resolution of the current output image, to ensure that it can be received and correctly parsed by the image receiving device, the obtained current operating state data needs to be encapsulated in a predetermined data format (such as hexadecimal numbers) to obtain an operating state data packet of the vehicle-mounted camera. During the encapsulation process, the length of the operating state data packet needs to be determined. The operating state data packet may include a packet header flag Data Head, the current operating state data, a packet tail flag TAIL, and a check code (such as CRC16 (Cyclic Redundancy Check 16, 16-bit cyclic redundancy check)), that is, the operating state data packet is: packet header flag + current operating state data (including real-time temperature data, real-time frame rate of the current output image, real-time total number of frames of the current output image, fault code FaultCode1 of the vehicle-mounted camera, etc.) + code of the vehicle-mounted camera + firmware version number + check code + packet tail flag, which can also be expressed as: Data Head + real-time temperature value + real-time frame rate + number of frames + FaultCode1 +... CRC16 + TAIL. Among them, the position of each data is fixed to facilitate the subsequent data parsing to identify the operating state data packet and verify the integrity of the data, and to confirm that no errors occur during the data transmission process.

[0081] It should be noted that the image transmission of in-vehicle cameras generally follows specific standards or protocols (such as MIPI (Mobile Industry Processor Interface) or DVP (Digital VideoPort)). These standards have clear regulations on the data volume of each frame of image. If the running status data packet is too long, it may exceed the maximum data volume that a frame of image can carry, resulting in data loss or transmission failure. Therefore, by constraining the total length of the running status data packet, it can be ensured that it can adapt to the existing image transmission framework. In the embodiments of the present application, the total length of the running status data packet ≤ the resolution of one line of the image (if the image resolution is expressed as N(H)×M(V), where if N = 1280 pixels, then the total length of the data packet ≤ 1280 bytes).

[0082] Furthermore, to improve the data feedback efficiency and reliability, in the embodiments of the present application, the current running status data is embedded in the last line of the image and the image transmission channel is reused.

[0083] Specifically, as Figure 3 and Figure 4 shown, assume that the normal image output resolution is N(H)×M(V), that is, the horizontal resolution is N pixels and there are M rows in total. To avoid reading one by one through I2C, realize the one-time feedback of all running status data, and at the same time realize the synchronous feedback of the image to meet the requirements of real-time monitoring in intelligent driving. When the camera processing chip generates each frame of image, a new line is automatically added at the end of the current output image's horizontal resolution to obtain the target resolution. Then, based on the target resolution, the embedding position of the running status data packet is determined. That is to say, the target resolution of the current output image is N(H)×(M + 1)(V), and the embedding position is the new line added to the horizontal resolution of the current output image, that is, the last line of the horizontal resolution of the current output image. Thus, after the running status data is encapsulated, the running status data packet is correspondingly embedded in the last line of the horizontal resolution of the current output image, and finally, the target output image with the running status data is obtained.

[0084] For example, assume that the current image output resolution is 1280×720 (N = 1280, M = 720), and the adjusted output target resolution is 1280×721, that is, the 721st line is the running status data packet encapsulated, and finally, the target output image with the running status data is obtained.

[0085] Thus, by automatically embedding the status data into the current output image and replacing the one-by-one reading through I2C, the strict synchronization between the status of the in-vehicle camera and the current output image frame can be achieved, thereby ensuring the data transmission efficiency and system stability.

[0086] Step S203 includes the following aspects:

[0087] Optionally, in an embodiment of the present application, before parsing the target output image by the image receiving device to obtain the current operating state data and the current output image, it further includes: determining whether to receive the verification failure information of the target output image of the image receiving device; if the verification failure information is received, retransmitting the target output image based on the verification failure information.

[0088] Specifically, after obtaining the target output image with the operating state data, the target output image can be transmitted back to the image receiving device in the MIPI or DVP manner, and the image receiving device parses and packs the operating state data embedded in the target output image to obtain the current operating state data and the current output image.

[0089] Specifically, as Figure 3 shown, after the image receiving device receives the target output image, the current output image (the first 720 lines) is normally displayed or processed, and at the same time, the last line (the 721st line) of the target output image, that is, the line where the operating state data is packed, is intercepted, unpacked according to the encapsulation protocol, and the header mark, the current operating state data, the tail mark, and the check code of the operating state data packet are parsed and verified to obtain the current output image and the parsed current operating state data. For example, check whether the header mark is 0xAA, whether the tail mark is 0x55, and use the CRC16 check code to confirm whether the data is damaged, and obtain information such as the temperature and frame rate of the in-vehicle camera in sequence.

[0090] It should be noted that during the data parsing process, if the verification fails, it is necessary to send the verification failure information of the target output image to the in-vehicle camera, which helps the in-vehicle camera to retransmit the target output image to the image receiving device in a timely manner when receiving the verification failure information, so as to ensure that the data can finally reach the image receiving device correctly and without error, thereby improving the reliability of the entire system.

[0091] Optionally, in an embodiment of the present application, after parsing the target output image by the image receiving device to obtain the current operating state data and the current output image, it further includes: determining whether the in-vehicle camera meets the preset fault handling conditions based on the current operating state data; if the in-vehicle camera meets the preset fault handling conditions, determining the fault handling strategy based on the current operating state data to perform fault handling according to the fault handling strategy.

[0092] Among them, the preset fault handling conditions can be set by those skilled in the art according to actual test requirements, and no specific limitation is made here.

[0093] Specifically, when parsing and verifying the packet header mark, current operating status data, packet tail mark, and checksum of the operating status data packet, if any abnormal operating status data is detected, it can be determined that the on-vehicle camera meets the preset fault handling conditions. Then, based on the current operating status data, a fault handling strategy is determined to perform fault handling according to the fault handling strategy.

[0094] For example, if it is detected that the temperature of the on-vehicle camera is too high, such as the temperature > 80°C, at this time, it is determined that the on-vehicle camera meets the preset fault handling conditions. Then, based on the current operating status data, a fault handling strategy is determined to perform fault handling according to the fault handling strategy. That is to say, when the temperature of the on-vehicle camera is too high, the temperature handling conditions are met at this time, and a corresponding temperature handling strategy is generated, such as sending a rest command to the on-vehicle camera to let the on-vehicle camera rest and cool down, or reducing the camera frame rate (for example, from 30fps → 15fps) to reduce heat generation, so as to prevent the on-vehicle camera from being damaged due to overheating or the image quality from deteriorating.

[0095] Furthermore, if it is detected that the real-time frame rate of the current output image is too low. For example, the output frame rate of the on-vehicle camera is 30fps, but the image receiving device analyzes and finds that the actual frame rate is only 18fps. At this time, it can be determined that the real-time frame rate of the current output image is lower than the threshold (for example, 24fps), that is, the on-vehicle camera meets the preset fault handling conditions. At this time, the signal quality of the MIPI interface can be detected to confirm whether the image processing is delayed due to the overload of the system CPU (Central Processing Unit), and the camera driver module can be restarted or the backup camera can be switched to recover, so as to ensure the real-time performance of the autonomous driving system and avoid the perception algorithm from failing due to insufficient frame rate.

[0096] Furthermore, when the image receiving device parses the operating status data packet and finds that the CRC check fails, it indicates that the data may be damaged due to transmission interference at this time, and it is determined that the current frame status data is not trustworthy. At this time, it can be determined that the on-vehicle camera meets the preset fault handling conditions, and the operating status data packet is marked as invalid. Then, the on-vehicle camera can be notified to resend the target output image through the I2C reverse channel. At the same time, the number of check failures can also be counted. If the failure occurs frequently, a hardware maintenance warning is triggered, which can prevent misjudgment caused by incorrect status data, improve the system robustness, and adapt to the complex electromagnetic environment of the vehicle.

[0097] Furthermore, if the image receiving device parses and finds that the synchronization signal is lost (such as fault code 0xE1), at this time, the fault code 0xE1 is extracted, and according to the fault code table, it is confirmed as a hardware synchronization exception. Then, a reset instruction can be sent through I2C to restart the camera ISP (Image Signal Processor) module. If the reset fails, the power supply of the vehicle-mounted camera is cut off and the backup device is switched, and the fault details are recorded, so as to quickly restore the camera function and reduce the system downtime.

[0098] It should be noted that after the above or other faults occur, the faults and solutions can be logged for subsequent diagnosis to improve the diagnosis accuracy.

[0099] Thus, by real-time monitoring the operating status of the vehicle-mounted camera (such as temperature, frame rate, resolution, etc.), potential problems or abnormal situations can be quickly identified. Once a problem is found, the corresponding fault handling strategy is immediately started, reducing the unexpected downtime, extending the service life of the device, improving the reliability and security of the system, and at the same time avoiding problems such as picture distortion and delay caused by hardware failures, thereby providing a more smooth and reliable user experience.

[0100] In summary, based on the discussion of the above specific embodiments, the embodiments of the present application can achieve the following beneficial effects:

[0101] (1) The operation status data packet is used as the image content and embedded in the last line of the image, and then the data packet that reflects its internal operation status is sent to the receiving end along with the image, thereby improving the data transmission efficiency and system stability;

[0102] (2) The operation status data is strictly synchronized with each frame of the image, and the existing MIPI / DVP image interface is reused without additional I2C communication, saving computing power resources;

[0103] (3) The data packet is built-in with CRC16 check, and at the same time, the operation status data is embedded in the image stream and parsed according to the protocol, so as to ensure the reliability of data transmission.

[0104] According to the method for transmitting vehicle-mounted camera status data according to an embodiment of the present application, the target resolution of the current output image is determined based on the obtained current operating status data of the vehicle-mounted camera and the resolution of the current output image, and the current operating status data is embedded into the current output image based on the target resolution to obtain a target output image. The target output image is transmitted back to the image receiving device of the vehicle, and the current operating status data and the current output image are obtained by parsing the target output image through the image receiving device. Thus, the problems caused by accessing and transmitting back images through communication interfaces such as I2C, such as low communication efficiency, susceptibility to external interference, occupation of system resources, and difficulty in meeting the requirements of real-time monitoring, are solved. By embedding the status data at the end of the image and reusing the image transmission channel, the data transmission efficiency and system stability are improved.

[0105] Next, a device for transmitting vehicle-mounted camera status data according to an embodiment of the present application will be described with reference to the accompanying drawings.

[0106] Figure 5 It is a block diagram of a device for transmitting vehicle-mounted camera status data according to an embodiment of the present application.

[0107] As Figure 5 shown, the device 10 for transmitting vehicle-mounted camera status data includes: an acquisition module 100, a data embedding module 200, and an analysis module 300.

[0108] Among them, the acquisition module 100 is used to acquire the current operating status data of the vehicle-mounted camera and the resolution of the current output image;

[0109] The data embedding module 200 is used to determine the target resolution of the current output image according to the current operating status data and the resolution of the current output image, and embed the current operating status data into the current output image based on the target resolution to obtain a target output image;

[0110] The analysis module 300 is used to transmit the target output image back to the image receiving device of the vehicle, and obtain the current operating status data and the current output image by parsing the target output image through the image receiving device.

[0111] Optionally, in an embodiment of the present application, the acquisition module 100 includes:

[0112] The first acquisition unit is used to acquire the dynamic operating status data and static performance data of the vehicle-mounted camera;

[0113] The second acquisition unit is used to obtain the current operating status data according to the dynamic operating status data and the static performance data;

[0114] Among them, the dynamic operating state data includes the real-time temperature data of the vehicle-mounted camera, the real-time frame rate of the currently output image, the real-time total number of frames of the currently output image, and the fault code of the vehicle-mounted camera, and the static performance data includes the code and firmware version number of the vehicle-mounted camera.

[0115] Optionally, in an embodiment of the present application, the data embedding module 200 includes:

[0116] A determination unit, configured to determine the packet header mark, packet tail mark, and check code of the current operating state data;

[0117] An encapsulation unit, configured to encapsulate the current operating state data based on the operating state data, packet header mark, packet tail mark, and check code to obtain an operating state data packet;

[0118] A data embedding unit, configured to determine the embedding position of the operating state data packet based on the target resolution, and based on the embedding position, correspondingly embed the operating state data packet into the currently output image to obtain a target output image.

[0119] Optionally, in an embodiment of the present application, after parsing the target output image by the image receiving device to obtain the current operating state data and the current output image, the parsing module 300 further includes:

[0120] A first judgment unit, configured to judge whether the vehicle-mounted camera meets a preset fault handling condition based on the current operating state data;

[0121] A fault handling unit, configured to, if the vehicle-mounted camera meets the preset fault handling condition, determine a fault handling strategy based on the current operating state data, so as to perform fault handling according to the fault handling strategy.

[0122] Optionally, in an embodiment of the present application, before parsing the target output image by the image receiving device to obtain the current operating state data and the current output image, the parsing module 300 further includes:

[0123] A second judgment unit, configured to judge whether to receive the check failure information of the target output image of the image receiving device;

[0124] A transmission unit, configured to, if the check failure information is received, retransmit the target output image based on the check failure information.

[0125] The device for transmitting vehicle-mounted camera status data according to the embodiments of the present application determines the target resolution of the current output image based on the currently obtained operating status data of the vehicle-mounted camera and the resolution of the current output image, embeds the currently obtained operating status data into the current output image based on the target resolution to obtain a target output image, transmits the target output image to the image receiving device of the vehicle, and the current operating status data and the current output image are obtained by parsing the target output image through the image receiving device. Thus, the problems such as low communication efficiency, susceptibility to external interference, occupation of system resources, and difficulty in meeting the requirements of real-time monitoring caused by accessing and transmitting images through communication interfaces such as I2C are solved. By embedding the status data at the end of the image and reusing the image transmission channel, the data transmission efficiency and system stability are improved.

[0126] Figure 6 The structural schematic diagram of the vehicle provided by the embodiments of the present application. The vehicle may include:

[0127] A memory 601, a processor 602, and a computer program stored on the memory 601 and executable on the processor 602.

[0128] When the processor 602 executes the program, it implements the method for transmitting vehicle-mounted camera status data provided in the above embodiments.

[0129] Further, the vehicle further includes:

[0130] A communication interface 603 for communication between the memory 601 and the processor 602.

[0131] The memory 601 is used for storing a computer program executable on the processor 602.

[0132] The memory 601 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory.

[0133] If the memory 601, the processor 602, and the communication interface 603 are independently implemented, the communication interface 603, the memory 601, and the processor 602 may be interconnected through a bus and communicate with each other. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6It is represented by only one thick line, but it does not mean that there is only one bus or one type of bus.

[0134] Optionally, in a specific implementation, if the memory 601, the processor 602, and the communication interface 603 are integrated on a single chip, the memory 601, the processor 602, and the communication interface 603 can communicate with each other through an internal interface.

[0135] The processor 602 may be a central processing unit (CPU for short), or an application specific integrated circuit (ASIC for short), or one or more integrated circuits configured to implement the embodiments of the present application.

[0136] This embodiment also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the above method for transmitting the in-vehicle camera status data is implemented.

[0137] This embodiment also provides a computer program product, including a computer program, and the computer program is executed to implement the method for transmitting the in-vehicle camera status data in the above embodiment.

[0138] In the description of this specification, the descriptions with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, without conflict, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0139] In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "N" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0140] Any process or method description represented in a flowchart or otherwise described herein can be understood to represent a module, segment, or portion of code including one or more executable instructions for implementing a customized logical function or process, and the scope of the preferred embodiments of the present application includes additional implementations, where functions may be executed in a substantially simultaneous manner or in a reverse order according to the functions involved, rather than in the order shown or discussed, which should be understood by those skilled in the art to which the embodiments of the present application pertain.

[0141] The logic and / or steps represented in a flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing a logical function, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or in conjunction with these instruction execution systems, apparatuses, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection portion (electronic device) having one or more wirings, a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, a computer-readable medium can even be paper or other suitable media on which a program can be printed, as the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or other appropriate processing as necessary, and then stored in a computer memory.

[0142] It should be understood that various parts of the present application can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application-specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0143] Those of ordinary skill in the art can understand that all or part of the steps carried out in the methods of the above embodiments can be completed by instructing relevant hardware through a program. The program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0144] In addition, in each of the embodiments of the present application, each functional unit can be integrated in a processing module, can exist separately physically for each unit, or two or more units can be integrated in a module. The above integrated module can be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0145] The storage medium mentioned above can be a read-only memory, a magnetic disk, an optical disc, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present application. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A method for transmitting back the status data of a vehicle-mounted camera, characterized in that, It includes the following steps: Obtain the current operating status data of the vehicle-mounted camera and the resolution of the currently output image; Determine the target resolution of the currently output image according to the current operating status data and the resolution of the currently output image, and embed the current operating status data into the currently output image based on the target resolution to obtain a target output image; Transmit the target output image back to the image receiving device of the vehicle, and parse the target output image through the image receiving device to obtain the current operating status data and the currently output image.

2. The method according to claim 1, wherein The obtaining of the current operating status data of the vehicle-mounted camera includes: Obtain the dynamic operating status data and static performance data of the vehicle-mounted camera; Obtain the current operating status data according to the dynamic operating status data and the static performance data; Wherein, the dynamic operating status data includes the real-time temperature data of the vehicle-mounted camera, the real-time frame rate of the currently output image, the total number of real-time frames of the currently output image, and the fault code of the vehicle-mounted camera, and the static performance data includes the code and firmware version number of the vehicle-mounted camera.

3. The method according to claim 1, wherein The embedding of the current operating status data into the currently output image based on the target resolution to obtain a target output image includes: Determine the packet header mark, packet tail mark and check code of the current operating status data; Package the current operating status data based on the operating status data, the packet header mark, the packet tail mark and the check code to obtain an operating status data packet; Based on the target resolution, determine the embedding position of the operating status data packet, and based on the embedding position, embed the operating status data packet correspondingly into the currently output image to obtain the target output image.

4. The method according to claim 1, wherein After parsing the target output image through the image receiving device to obtain the current operating status data and the currently output image, it further includes: Based on the current operating status data, determine whether the vehicle-mounted camera meets the preset fault handling conditions; If the vehicle-mounted camera meets the preset fault handling conditions, determine a fault handling strategy based on the current operating status data to perform fault handling according to the fault handling strategy.

5. The method according to claim 4, characterized in that Before parsing the target output image through the image receiving device to obtain the current operating status data and the currently output image, it further includes: Determine whether to receive the check failure information of the target output image of the image receiving device; If the check failure information is received, retransmit the target output image based on the check failure information.

6. A device for transmitting back the status data of a vehicle-mounted camera, characterized in that, It includes: An obtaining module, configured to obtain the current operating status data of the vehicle-mounted camera and the resolution of the currently output image; A data embedding module, configured to determine the target resolution of the currently output image according to the current operating status data and the resolution of the currently output image, and embed the current operating status data into the currently output image based on the target resolution to obtain a target output image; A parsing module, configured to send the target output image back to an image receiving device of the vehicle, and parse the target output image through the image receiving device to obtain the current operating state data and the current output image.

7. The device according to claim 6, characterized in that The obtaining module includes: A first obtaining unit, configured to obtain dynamic operating state data and static performance data of the in-vehicle camera; A second obtaining unit, configured to obtain the current operating state data according to the dynamic operating state data and the static performance data; Wherein, the dynamic operating state data includes real-time temperature data of the in-vehicle camera, real-time frame rate of the current output image, real-time total number of frames of the current output image, and fault codes of the in-vehicle camera, and the static performance data includes the code and firmware version number of the in-vehicle camera.

8. The device according to claim 6, characterized in that The data embedding module includes: A determining unit, configured to determine a packet header flag, a packet tail flag, and a check code of the current operating state data; An encapsulating unit, configured to encapsulate the current operating state data based on the operating state data, the packet header flag, the packet tail flag, and the check code to obtain an operating state data packet; A data embedding unit, configured to determine an embedding position of the operating state data packet based on the target resolution, and embed the operating state data packet into the current output image correspondingly based on the embedding position to obtain the target output image.

9. A vehicle, characterized in that, It includes: A memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor executes the program to implement the method for transmitting in-vehicle camera state data according to any one of claims 1-5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to be used for implementing the method for transmitting in-vehicle camera state data according to any one of claims 1-5.

Images