Autonomous driving system, method, device and storage medium

Abstract

The present application relates to the technical field of automatic driving, and especially relates to an automatic driving system, method, device and storage medium. The automatic driving system comprises a visual sensing module, a visual sensing data processing module, a first driving processing module and a second driving processing module; wherein the first driving processing module and the second driving processing module are mutually detected; the visual sensing module is used for collecting environmental data in the driving process of a vehicle; the visual sensing data processing module is used for performing branch deserializing processing on the environmental data, obtaining a first visual signal and a second visual signal, and outputting the first visual signal and the second visual signal to the first driving processing module and the second driving processing module respectively; when it is detected that one driving processing module is abnormal, the other driving processing module performs automatic driving takeover and automatic driving decision processing; thereby the reliability of the automatic driving system when a part of the automatic driving system has a problem is improved.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to an autonomous driving system, method, device and storage medium. Background Technology

[0002] With the development of autonomous driving technology, improving the reliability of autonomous driving systems has become an issue that needs to be addressed.

[0003] In existing technologies, the overall reliability of an autonomous driving system is usually improved by enhancing the reliability of each component, but there is a lack of solutions to improve reliability when problems occur in autonomous driving.

[0004] Therefore, there is an urgent need to provide a solution that can improve the reliability of an autonomous driving system when a part of the system malfunctions. Summary of the Invention

[0005] The main objective of this application is to provide an autonomous driving system, method, device, and storage medium to address the problem of low reliability in existing autonomous driving systems when a part of the system malfunctions.

[0006] The first aspect of this invention provides an autonomous driving system, comprising: a vision sensing module, a vision sensing data processing module, a first driving processing module, and a second driving processing module; wherein the first driving processing module and the second driving processing module perform mutual detection; the vision sensing module is used to collect environmental data during vehicle driving; the vision sensing data processing module is used to perform deserialization processing on the environmental data to obtain a first visual signal and a second visual signal, and outputs them to the first driving processing module and the second driving processing module respectively; when the first driving processing module detects an anomaly in the second driving processing module, the first driving processing module takes over autonomous driving and performs autonomous driving decision processing based on the first visual signal; when the second driving processing module detects an anomaly in the first driving processing module, the second driving processing module takes over autonomous driving and performs autonomous driving decision processing based on the second visual signal.

[0007] Optionally, in a first implementation of the first aspect of the present invention, the visual sensing data processing module includes a first deserializer and a preprocessor; the first deserializer is used to deserialize the environmental data to obtain parallel data, and transmit the parallel data to the preprocessor and the second driving processing module respectively; the preprocessor is used to receive the parallel data sent from the first deserializer, and perform format conversion and noise reduction on the parallel data to obtain preprocessed data; the preprocessor is also used to transmit the preprocessed data to the first driving processing module.

[0008] Optionally, in a second implementation of the first aspect of the present invention, the visual sensing data processing module further includes a serializer and a second deserializer; the serializer is used to serialize the environmental data to obtain serial data and transmit it to the second deserializer; the second deserializer is used to deserialize the serial data to obtain second parallel data and transmit the second parallel data to the second driving processing module.

[0009] Optionally, in a third implementation of the first aspect of the present invention, the first driving processing module is used to receive the preprocessed data and perform autonomous driving decision processing based on the preprocessed data; the second driving processing module is used to receive the second parallel data and perform autonomous driving decision processing based on the second parallel data.

[0010] Optionally, in a fourth implementation of the first aspect of the present invention, the first driving processing module is further configured to receive a second heartbeat signal sent from the second driving processing module, and determine whether the second heartbeat signal meets a preset second heartbeat condition; if the second heartbeat signal does not meet the preset second heartbeat condition, then autonomous driving takeover is performed, and autonomous driving decision processing is performed based on the preprocessed data; the second driving processing module is further configured to receive a first heartbeat signal sent from the first driving processing module, and determine whether the first heartbeat signal meets a preset first heartbeat condition; if the first heartbeat signal does not meet the preset first heartbeat condition, then autonomous driving takeover is performed, and autonomous driving decision processing is performed based on the second parallel data.

[0011] A second aspect of the present invention provides an autonomous driving method applied to an autonomous driving system, the autonomous driving system including a first driving processing module and a second driving processing module, the autonomous driving method including: collecting environmental data during vehicle driving; performing deserialization processing on the environmental data to obtain a first visual signal and a second visual signal, and outputting them to the first driving processing module and the second driving processing module respectively; detecting whether there is an anomaly in the first driving processing module and the second driving processing module; if an anomaly is detected in the first driving processing module, controlling the second driving processing module to take over autonomous driving and performing autonomous driving decision processing based on the second visual signal; if an anomaly is detected in the second driving processing module, controlling the first driving processing module to take over autonomous driving and performing autonomous driving decision processing based on the first visual signal.

[0012] Optionally, in a first implementation of the second aspect of the present invention, the step of performing deserialization processing on the environmental data to obtain a first visual signal and a second visual signal includes: performing double-rate sampling on the environmental data to obtain sampled data; outputting the sampled data through a preset clock cycle to obtain parallel data; and performing deserialized transmission of the parallel data through two preset data transmission lines to obtain the first visual signal and the second visual signal.

[0013] Optionally, in a second implementation of the second aspect of the present invention, detecting whether the first driving processing module and the second driving processing module are abnormal includes: receiving a second heartbeat signal sent from the second driving processing module through the first driving processing module, and determining whether the second heartbeat signal meets a preset second heartbeat condition through the first driving processing module; if the second heartbeat signal does not meet the preset second heartbeat condition, then the second driving processing module is abnormal; if the second heartbeat signal meets the preset second heartbeat condition, then the second driving processing module is not abnormal; receiving a first heartbeat signal sent from the first driving processing module through the second driving processing module, and determining whether the first heartbeat signal meets a preset first heartbeat condition through the second driving processing module; if the first heartbeat signal does not meet the preset first heartbeat condition, then the first driving processing module is abnormal; if the first heartbeat signal meets the preset first heartbeat condition, then the first driving processing module is not abnormal.

[0014] A third aspect of the present invention provides a computer device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor invokes the instructions in the memory to cause the computer device to perform the various steps of the above-described autonomous driving method.

[0015] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the steps of the above-described autonomous driving method.

[0016] This application proposes an autonomous driving system, method, device, and storage medium. The autonomous driving system includes a vision sensing module, a vision sensing data processing module, a first driving processing module, and a second driving processing module. The first and second driving processing modules perform mutual detection. The vision sensing module collects environmental data during vehicle driving. The vision sensing data processing module performs deserialization and deserialization processing on the environmental data to obtain a first visual signal and a second visual signal, which are then output to the first and second driving processing modules, respectively. When an anomaly is detected in one driving processing module, the other driving processing module takes over autonomous driving and performs autonomous driving decision processing, thereby improving the reliability of the autonomous driving system when a part of it malfunctions. By collecting environmental data during vehicle driving; performing deserialization on the environmental data to obtain a first visual signal and a second visual signal, and outputting them to the first driving processing module and the second driving processing module respectively; detecting whether there are any abnormalities in the first driving processing module and the second driving processing module; if an abnormality is detected in the first driving processing module, controlling the second driving processing module to take over autonomous driving and making autonomous driving decisions based on the second visual signal; if an abnormality is detected in the second driving processing module, controlling the first driving processing module to take over autonomous driving and making autonomous driving decisions based on the first visual signal, this solves the problem of low reliability when a part of the autonomous driving system malfunctions in the prior art. Attached Figure Description

[0017] Figure 1 This is a first structural schematic diagram of the autonomous driving system in the first embodiment of the present invention;

[0018] Figure 2 This is a second structural schematic diagram of the autonomous driving system in the second embodiment of the present invention;

[0019] Figure 3 This is a third structural schematic diagram of the autonomous driving system in the third embodiment of the present invention;

[0020] Figure 4 This is a schematic diagram of one embodiment of the autonomous driving method in this invention;

[0021] Figure 5 This is a schematic diagram of another embodiment of the autonomous driving method in this invention;

[0022] Figure 6 This is a schematic diagram of one embodiment of the computer device in this invention. Detailed Implementation

[0023] To address the issue of low reliability in existing autonomous driving systems when a part of the system malfunctions, this application provides an autonomous driving system, method, apparatus, and storage medium. This method collects environmental data during vehicle driving; performs deserialization processing on the environmental data to obtain a first visual signal and a second visual signal, which are then output to the first driving processing module and the second driving processing module, respectively; detects whether there are any abnormalities in the first driving processing module and the second driving processing module; if an abnormality is detected in the first driving processing module, the second driving processing module is controlled to take over autonomous driving and make autonomous driving decisions based on the second visual signal; if an abnormality is detected in the second driving processing module, the first driving processing module is controlled to take over autonomous driving and make autonomous driving decisions based on the first visual signal. By converting the captured optical signal into a digital signal and transmitting the digital signal to a serializer for serialization processing, the data is organized into packets and then sent serially via a coaxial cable, increasing the information transmission rate. Furthermore, a pre-defined deserializer in the autonomous driving system deserializes the serial signal to obtain parallel signals, which are then output to the first driving processing module and the second driving processing module for autonomous driving decision processing. This solves the problem of low reliability in existing autonomous driving systems when a part of the system malfunctions.

[0024] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” or “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 The first structural schematic diagram of the first embodiment of the autonomous driving system in this invention includes:

[0026] The system includes a visual sensing module 10, a visual sensing data processing module 20, a first driving processing module 30, and a second driving processing module 40; wherein the first driving processing module 30 and the second driving processing module 40 perform mutual detection.

[0027] The visual sensing module 10 is used to collect environmental data during vehicle driving;

[0028] The visual sensing data processing module 20 is used to perform deserialization processing on the environmental data to obtain a first visual signal and a second visual signal, and output them to the first driving processing module 30 and the second driving processing module 40 respectively.

[0029] When the first driving processing module 30 detects an abnormality in the second driving processing module 40, the first driving processing module 30 takes over autonomous driving and makes autonomous driving decision-making based on the first visual signal.

[0030] When the second driving processing module 40 detects an abnormality in the first driving processing module 30, the second driving processing module 40 takes over autonomous driving and makes autonomous driving decisions based on the second visual signal.

[0031] In practical applications, the visual sensing module 10 can be a camera;

[0032] The environmental data includes data in GMSL format, that is, data that meets the GMSL (Gigabit Multimedia Serial Link) serial transmission interface standard.

[0033] The first visual signal and the second visual signal can be modulated into MIPI format signals, that is, signals that satisfy the Mobile Industry Processor Interface (MIPI) alliance protocol.

[0034] Please see Figure 2 The second structural diagram of the autonomous driving system in the second embodiment of the present invention shows that the visual sensing data processing module 20 includes a first deserializer 201 and a preprocessor 202.

[0035] The first deserializer 201 is used to deserialize the environmental data to obtain parallel data, and transmit it to the preprocessor 202 and the second driving processing module 40 respectively.

[0036] The preprocessor 202 is used to receive parallel data sent from the first deserializer 201, and to perform format conversion and noise reduction on the parallel data to obtain preprocessed data.

[0037] The preprocessor 202 is also used to transmit the preprocessed data to the first driving processing module 30.

[0038] In practical applications, the parallel data includes data in MIPI format, that is, data that meets the Mobile Industry Processor Interface (MIPI) alliance protocol.

[0039] Please see Figure 3 The third structural diagram of the autonomous driving system in the third embodiment of the present invention shows that the visual sensing data processing module 20 includes a first deserializer 201, a preprocessor 202, a serializer 203, and a second deserializer 204.

[0040] The serializer 203 is used to serialize the environmental data to obtain serial data, and then transmit it to the second deserializer 204.

[0041] The second deserializer 204 is used to deserialize the serial data to obtain the second parallel data, and transmit the second parallel data to the second driving processing module 40;

[0042] The second driving processing module 40 is used to receive the second parallel data and perform autonomous driving decision processing based on the second parallel data;

[0043] The first driving processing module 30 is also used to receive a second heartbeat signal sent from the second driving processing module 40, and to determine whether the second heartbeat signal meets the preset second heartbeat condition;

[0044] If the second heartbeat signal does not meet the preset second heartbeat condition, then autonomous driving takeover will be initiated, and autonomous driving decision processing will be performed based on the preprocessed data.

[0045] The second driving processing module 40 is also used to receive a first heartbeat signal sent from the first driving processing module 30, and to determine whether the first heartbeat signal meets a preset first heartbeat condition;

[0046] If the first heartbeat signal does not meet the preset first heartbeat condition, then autonomous driving takeover will be initiated, and autonomous driving decision processing will be performed based on the second parallel data.

[0047] In practical applications, the serial data includes data in GMSL format, that is, data that meets the GMSL (Gigabit Multimedia Serial Link) serial transmission interface standard.

[0048] The second parallel data includes data in MIPI format, which is data that meets the Mobile Industry Processor Interface (MIPI) alliance protocol.

[0049] In practical applications, a backup module can also be configured between the serializer 203 and the second deserializer 204, wherein the backup module has a memory;

[0050] The backup module is used to receive serial data sent by the serializer 203, copy the serial data, and save the copied data to the memory.

[0051] In summary, the autonomous driving system includes a vision sensing module, a vision sensing data processing module, a first driving processing module, and a second driving processing module. The first and second driving processing modules perform mutual detection. The vision sensing module collects environmental data during vehicle driving. The vision sensing data processing module performs deserialization processing on the environmental data to obtain a first visual signal and a second visual signal, which are then output to the first and second driving processing modules, respectively. When an anomaly is detected in one driving processing module, the other driving processing module takes over autonomous driving and performs autonomous driving decision-making. Compared to existing technologies, this application configures a deserializer and a serializer in the vision sensing data processing module. The deserializer deserializes the data sent from the vision sensing module and transmits it in two separate paths to the preprocessing module and the serializer module. The preprocessing module preprocesses the data before sending it to the first driving processing module for processing. The serializer re-serializes the data, and the second serializer deserializes it, facilitating data transmission and achieving redundant backup of the vision sensing data. This improves the reliability of the autonomous driving system when a problem occurs.

[0052] Please see Figure 4 One embodiment of the autonomous driving method in this invention includes the following implementation steps:

[0053] 401. Collect environmental data during vehicle driving;

[0054] In this step, the process of collecting environmental data during vehicle driving is based on a preset autonomous driving system, wherein the autonomous driving system has at least one camera;

[0055] This step can be implemented in the following way:

[0056] The camera collects environmental data during the vehicle's driving process.

[0057] In practical applications, the camera includes an image sensor.

[0058] The process of collecting environmental data during vehicle driving also includes:

[0059] The captured light signals are converted into digital signals by an image sensor, wherein the digital signals include environmental data information, pixel clock, line synchronization signal and frame synchronization signal;

[0060] The digital signal is transmitted to the serializer, for example, based on the CSI2 protocol;

[0061] After receiving the data, the serializer serializes the information, organizes the data into packets, and then sends the packets out serially through a coaxial cable to obtain environmental data. There are two directions of data flow transmission when passing through the coaxial cable. One is a forward channel, which is used to send the environmental data captured by the camera, with a bandwidth of 6Gbps. The other is a feedback channel, which is used to receive response information, with a bandwidth of 187Mbps.

[0062] 402. Perform deserialization on the environmental data to obtain the first visual signal and the second visual signal, and output them to the first driving processing module and the second driving processing module respectively.

[0063] In this step, the process of deserializing environmental data is based on a preset autonomous driving system, wherein the autonomous driving system has a deserializer;

[0064] This step can be implemented in the following way:

[0065] The environmental data is deserialized using the deserializer.

[0066] The deserialized data is split into two signals to obtain a first visual signal and a second visual signal, which are then transmitted to the first driving processing module and the second driving processing module, respectively. For example, the signals can be transmitted via a coaxial cable.

[0067] In practical applications, this step also includes:

[0068] The serial data packet is deserialized to obtain the second digital signal;

[0069] The second digital signal is transmitted to the first driving processing module and the second driving processing module respectively.

[0070] 403. Check if there are any abnormalities in the first driving processing module and the second driving processing module;

[0071] This step can be implemented in the following way:

[0072] The first driving processing module detects whether the second driving processing module has any abnormalities.

[0073] The second driving processing module detects whether the first driving processing module has any abnormalities.

[0074] 404. If an abnormality is detected in the first driving processing module, the second driving processing module is controlled to take over autonomous driving and autonomous driving decision processing is performed based on the second visual signal.

[0075] This step is based on an autonomous driving system, which includes a driving control module;

[0076] This step can be implemented in the following way:

[0077] If an anomaly is detected in the first driving processing module, the second driving processing module will perform autonomous driving decision processing based on the second visual signal to obtain a second processing result.

[0078] The autonomous driving decision is made based on the second processing result. For example, a second control signal is generated based on the second processing result, the second control signal is sent to the driving control module, and the driving control module is controlled according to the second control signal.

[0079] 405. If an abnormality is detected in the second driving processing module, the first driving processing module is controlled to take over autonomous driving and make autonomous driving decision-making based on the first visual signal.

[0080] This step is based on an autonomous driving system, which includes a driving control module;

[0081] This step can be implemented in the following way:

[0082] If an anomaly is detected in the second driving processing module, the first driving processing module performs autonomous driving decision processing based on the first visual signal to obtain the first processing result.

[0083] The autonomous driving decision is made based on the first processing result. For example, a first control signal is generated based on the first processing result, the second control signal is sent to the driving control module, and the driving control module is controlled based on the first control signal.

[0084] By implementing the above method, environmental data during vehicle driving is collected; the environmental data is deserialized to obtain a first visual signal and a second visual signal, which are then output to the first driving processing module and the second driving processing module, respectively; the presence of abnormalities in the first driving processing module and the second driving processing module is detected; if an abnormality is detected in the first driving processing module, the second driving processing module is controlled to take over autonomous driving and make autonomous driving decisions based on the second visual signal; if an abnormality is detected in the second driving processing module, the first driving processing module is controlled to take over autonomous driving and make autonomous driving decisions based on the first visual signal; above, by converting the captured light signal into a digital signal and transmitting the digital signal to a serializer to serialize the information, the data is organized into packets, and then the packets are sent out serially through a coaxial cable, improving the information transmission rate. Furthermore, the serial signal is deserialized by a pre-set deserializer in the autonomous driving system to obtain parallel signals, which are then output to the first driving processing module and the second driving processing module for autonomous driving decision processing, thereby solving the problem of low reliability in existing autonomous driving systems when a part of the system malfunctions.

[0085] Please see Figure 5 Another embodiment of the autonomous driving method in this invention includes the following implementation steps:

[0086] 501. Collect environmental data during vehicle driving;

[0087] This step is based on a preset autonomous driving system, wherein the autonomous driving system includes a visual sensing module;

[0088] In this step, the visual sensing module includes a visible light sensor, an infrared sensor, an ultraviolet sensor, and a laser rangefinder.

[0089] This step can be implemented in the following way:

[0090] The system receives visual sensing data collected by the visual sensing module, wherein the sensing data includes data collected by a visible light sensor, data collected by an infrared sensor, data collected by an ultraviolet sensor, and data collected by a laser rangefinder.

[0091] The visual sensing data is converted into environmental data, wherein the environmental data...

[0092] 502. Perform deserialization on the environmental data to obtain the first visual signal and the second visual signal, and output them to the first driving processing module and the second driving processing module respectively.

[0093] This step can be implemented in the following way:

[0094] The environmental data is sampled to obtain sampled data. For example, sampling can be performed using double-rate sampling or multi-rate sampling.

[0095] The sampled data is output through a preset clock cycle to obtain parallel data. For example, a signal cycle can be set as the clock cycle.

[0096] The parallel data is split and transmitted through two preset data transmission lines to obtain the first visual signal and the second visual signal. For example, the data transmission lines include a first data transmission line that connects the deserializer and the first driving processing module through a coaxial cable and a second data transmission line that connects the deserializer and the second driving processing module through a coaxial cable.

[0097] In practical applications, this step can be implemented using a deserializer, such as the DS90UB948 / MAX96762 deserializer.

[0098] 503. Check if there are any abnormalities in the first driving processing module and the second driving processing module;

[0099] This step can be implemented in the following way:

[0100] The first driving processing module receives a second heartbeat signal sent from the second driving processing module, and the first driving processing module determines whether the second heartbeat signal meets a preset second heartbeat condition. For example, the second heartbeat condition can be set to the sending period of the second heartbeat signal not being greater than a preset time threshold, and the time threshold can be set to 0.01 seconds.

[0101] If the second heartbeat signal does not meet the preset second heartbeat condition, then the second driving processing module is abnormal;

[0102] If the second heartbeat signal meets the preset second heartbeat condition, then the second driving processing module is not abnormal;

[0103] The second driving processing module receives the first heartbeat signal sent from the first driving processing module and determines whether the first heartbeat signal meets a preset first heartbeat condition. For example, the first heartbeat condition can be set to the transmission period of the first heartbeat signal not being greater than a preset time threshold, which can be set to 0.01 seconds.

[0104] If the first heartbeat signal does not meet the preset first heartbeat condition, then the first driving processing module is abnormal;

[0105] If the first heartbeat signal meets the preset first heartbeat condition, then the first driving processing module is not abnormal.

[0106] In practical applications, the first heartbeat signal and the second heartbeat signal can be respectively set as the heartbeat data packet corresponding to the first driving processing module and the heartbeat data packet corresponding to the second driving processing module.

[0107] In practical applications, the process of detecting whether there is an abnormality in the first driving processing module and the second driving processing module also includes performing heartbeat detection on the connection between the first driving processing module and the second driving processing module respectively.

[0108] Specifically, the process of performing heartbeat detection on the connection between the first driving processing module and the second driving processing module includes:

[0109] In the heartbeat detection process initiated by the first driving processing module, the first driving processing module and the second driving processing module mark the heartbeat detection process messages sent in the order of sending. The first driving processing module determines whether the heartbeat detection response is a response to the last heartbeat detection request sent by the first driving processing module based on the identifier in the received heartbeat detection response.

[0110] If so, then determine that the response sequence number in this response is greater than the request sequence number it maintains;

[0111] If it is greater than 1, then it is determined that the second driving processing module does not have any abnormalities.

[0112] If it is not greater than, then the second driving processing module is judged to be abnormal;

[0113] For the heartbeat detection process initiated by the first driving processing module, the first driving processing module maintains a request sequence number and identifies this request sequence number in the heartbeat detection request it sends. The second driving processing module maintains a response sequence number and identifies this response sequence number in the heartbeat detection response it sends. During initialization, the request sequence number is less than or equal to the response sequence number.

[0114] After receiving the heartbeat detection request, the second driving processing module determines that if the request sequence number in the request is less than or equal to the maintenance response sequence number, it increments the response sequence number by a preset interval value and marks the updated response sequence number in the heartbeat detection response returned to the first driving processing module.

[0115] After receiving a heartbeat detection response within a preset heartbeat detection time interval, the first driving processing module determines that the response sequence number in this response is greater than the request sequence number it maintains. If so, it updates the request sequence number to the value of this response sequence number and identifies this updated request sequence number in the heartbeat detection request sent in the next heartbeat detection cycle.

[0116] In practical applications, the process of detecting whether there is an abnormality in the first driving processing module and the second driving processing module also includes determining whether there is an abnormality in the connection between the first driving processing module and the second driving processing module.

[0117] Specifically, determining whether there is an anomaly in the connection between the first driving processing module and the second driving processing module includes:

[0118] Upon receiving a heartbeat packet from the second driving processing module, query whether the first driving processing module connected to the second driving processing module stores the heartbeat detection record of the second driving processing module;

[0119] If the first driving processing module stores the heartbeat detection record of the second driving processing module in its memory, then the information recorded in the heartbeat detection record is updated according to the received heartbeat packet;

[0120] The heartbeat detection records stored in the first driving processing module are polled at a preset frequency to determine whether there are any abnormal heartbeat detection records in the heartbeat detection records. For example, the preset frequency can be set to once per second.

[0121] If the abnormal heartbeat detection record exists, it is determined that there is an abnormality in the connection between the first driving processing module and the second driving processing module;

[0122] If no abnormal heartbeat detection record is found, it is determined that there is no abnormality in the connection between the first driving processing module and the second driving processing module.

[0123] 504. If an abnormality is detected in the first driving processing module, the second driving processing module is controlled to take over autonomous driving and autonomous driving decision-making is performed based on the second visual signal.

[0124] In practical applications, before making autonomous driving decisions based on second-vision signals, the following steps are also included:

[0125] The second visual signal is serialized using a preset serializer;

[0126] The serialized data is transmitted to a preset second deserializer;

[0127] The serialized data is deserialized using a pre-set second deserializer.

[0128] The second driving processing module performs autonomous driving decision processing based on the deserialized data.

[0129] 505. If an abnormality is detected in the second driving processing module, the first driving processing module is controlled to take over autonomous driving and make autonomous driving decision-making based on the first visual signal.

[0130] In practical applications, before making autonomous driving decisions based on the first visual signal, the following steps are also included:

[0131] The first visual signal is converted into a preset format to obtain the converted environmental data. For example, the first visual signal is converted into an RGB image.

[0132] The transformed environmental data is denoised using a preset denoising algorithm. For example, the denoising can be performed using mean filtering, median filtering, and Wiener filtering.

[0133] By implementing the above method, environmental data during vehicle driving is collected; the environmental data is deserialized to obtain a first visual signal and a second visual signal, which are then output to the first driving processing module and the second driving processing module, respectively; the presence of abnormalities in the first driving processing module and the second driving processing module is detected; if an abnormality is detected in the first driving processing module, the second driving processing module is controlled to take over autonomous driving and make autonomous driving decisions based on the second visual signal; if an abnormality is detected in the second driving processing module, the first driving processing module is controlled to take over autonomous driving and make autonomous driving decisions based on the first visual signal; above, by converting the captured light signal into a digital signal and transmitting the digital signal to a serializer to serialize the information, the data is organized into packets, and then the packets are sent out serially through a coaxial cable, improving the information transmission rate. Furthermore, the serial signal is deserialized by a pre-set deserializer in the autonomous driving system to obtain parallel signals, which are then output to the first driving processing module and the second driving processing module for autonomous driving decision processing, thereby solving the problem of low reliability in existing autonomous driving systems when a part of the system malfunctions.

[0134] Please see Figure 6 The following is a detailed description of one embodiment of the computer device in this invention from the perspective of hardware processing.

[0135] Figure 6This is a schematic diagram of the structure of a computer device 600 provided in an embodiment of the present invention. The computer device 600 can vary significantly due to different configurations or performance characteristics. It may include one or more central processing units (CPUs) 610 (e.g., one or more processors) and a memory 620, and one or more storage media 630 (e.g., one or more mass storage devices) for storing application programs 633 or data 632. The memory 620 and storage media 630 can be temporary or persistent storage. The program stored in the storage media 630 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the computer device 600. Furthermore, the processor 610 may be configured to communicate with the storage media 630 and execute the series of instruction operations in the storage media 630 on the computer device 600.

[0136] Computer device 600 may also include one or more power supplies 640, one or more wired or wireless network interfaces 650, one or more input / output interfaces 660, and / or one or more operating systems 631, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 6 The computer device structure shown does not constitute a limitation on the computer device provided in this application. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0137] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of the above-described automatic driving method.

[0138] In practical applications, the methods described above can be implemented based on artificial intelligence (AI) technology. AI is the theory, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. Specifically, it can be executed on a server. The server can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and AI platforms.

[0139] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described apparatus and unit can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0140] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0141] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An autonomous driving system, characterized in that, The autonomous driving system includes a vision sensing module, a vision sensing data processing module, a first driving processing module, and a second driving processing module; the vision sensing data processing module includes a first deserializer and a preprocessor; wherein the first driving processing module and the second driving processing module perform mutual detection; The visual sensing module is used to collect environmental data during vehicle driving. The visual sensing data processing module is used to perform deserialization processing on the environmental data to obtain a first visual signal and a second visual signal, and output them to the first driving processing module and the second driving processing module, respectively. This includes: the first deserializer deserializing the environmental data to obtain parallel data, which is then transmitted to the preprocessor and the second driving processing module; the preprocessor receiving the parallel data from the first deserializer and performing format conversion and noise reduction on the parallel data to obtain preprocessed data; and the preprocessor transmitting the preprocessed data to the first driving processing module. When the first driving processing module detects an abnormality in the second driving processing module, the first driving processing module takes over autonomous driving and makes autonomous driving decisions based on the first visual signal. When the second driving processing module detects an anomaly in the first driving processing module, the second driving processing module takes over autonomous driving and makes autonomous driving decisions based on the second visual signal.

2. The autonomous driving system according to claim 1, characterized in that, The visual sensing data processing module also includes a serializer and a second deserializer. The serializer is used to serialize the environmental data to obtain serial data, and then transmit it to the second deserializer. The second deserializer is used to deserialize the serial data to obtain second parallel data, and then transmit the second parallel data to the second driving processing module.

3. The autonomous driving system according to claim 2, characterized in that, The first driving processing module is used to receive the preprocessed data and perform autonomous driving decision processing based on the preprocessed data; The second driving processing module is used to receive the second parallel data and perform autonomous driving decision processing based on the second parallel data.

4. The autonomous driving system according to claim 3, characterized in that, The first driving processing module is also used to receive a second heartbeat signal sent from the second driving processing module, and to determine whether the second heartbeat signal meets the preset second heartbeat condition; If the second heartbeat signal does not meet the preset second heartbeat condition, then autonomous driving takeover will be initiated, and autonomous driving decision processing will be performed based on the preprocessed data. The second driving processing module is also used to receive a first heartbeat signal sent from the first driving processing module, and to determine whether the first heartbeat signal meets a preset first heartbeat condition; If the first heartbeat signal does not meet the preset first heartbeat condition, then autonomous driving takeover will be initiated, and autonomous driving decision processing will be performed based on the second parallel data.

5. An autonomous driving method, applied to an autonomous driving system, characterized in that, The autonomous driving system includes a first driving processing module and a second driving processing module, and the autonomous driving method includes: Collect environmental data during vehicle driving; The environmental data is deserialized to obtain a first visual signal and a second visual signal, which are then output to the first driving processing module and the second driving processing module, respectively. Detect whether there are any abnormalities in the first driving processing module and the second driving processing module; If an anomaly is detected in the first driving processing module, the second driving processing module is controlled to take over autonomous driving and make autonomous driving decision-making based on the second visual signal. If an anomaly is detected in the second driving processing module, the first driving processing module is controlled to take over autonomous driving and make autonomous driving decisions based on the first visual signal.

6. The autonomous driving method according to claim 5, characterized in that, The step of deserializing the environmental data to obtain a first visual signal and a second visual signal includes: The environmental data is double-sampled to obtain sampled data; The sampled data is output through a preset clock cycle to obtain parallel data; The parallel data is split and transmitted through two preset data transmission lines to obtain the first visual signal and the second visual signal.

7. The autonomous driving method according to claim 5, characterized in that, The detection of whether the first driving processing module and the second driving processing module are abnormal includes: The first driving processing module receives the second heartbeat signal sent from the second driving processing module, and the first driving processing module determines whether the second heartbeat signal meets the preset second heartbeat condition. If the second heartbeat signal does not meet the preset second heartbeat condition, then the second driving processing module is abnormal; If the second heartbeat signal meets the preset second heartbeat condition, then the second driving processing module is not abnormal; The second driving processing module receives the first heartbeat signal sent from the first driving processing module and determines whether the first heartbeat signal meets the preset first heartbeat condition. If the first heartbeat signal does not meet the preset first heartbeat condition, then the first driving processing module is abnormal; If the first heartbeat signal meets the preset first heartbeat condition, then the first driving processing module is not abnormal.

8. A computer device, characterized in that, include: A memory and at least one processor, wherein the memory stores instructions and the memory and the at least one processor are interconnected via a circuit; The at least one processor invokes the instructions in the memory to cause the computer device to perform the steps of the autonomous driving method as described in any one of claims 5-7.

9. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by a processor, it implements the various steps of the autonomous driving method as described in any one of claims 5-7.

Images