Auxiliary driving system supporting image desensitization, auxiliary driving control method and vehicle

Abstract

The embodiment of the invention discloses an auxiliary driving system supporting image desensitization, an auxiliary driving control method and a vehicle, relates to the technical field of auxiliary driving, and not only can perform image desensitization processing, but also can effectively reduce the cost of data acquisition equipment. The system comprises a camera module used for obtaining first original image data; the image data copying unit is used for obtaining first copied image data and second copied image data from the camera module; the auxiliary driving unit is connected with the image data copying unit and is used for performing first image processing on the first copied image data to obtain first YUV image data and desensitizing the first YUV image data to obtain desensitized area information; and the data acquisition unit is connected with the image data copying unit and the auxiliary driving unit, performs second image processing on the second copied image data to obtain second YUV image data, and performs desensitization processing on the second YUV image data by using the desensitization area information. The method is suitable for the image desensitization operation of the auxiliary driving system.

Description

Technical Field

[0001] The present invention relates to the field of assisted driving technology, and in particular to an assisted driving system supporting image desensitization, an assisted driving control method and a vehicle. Background Art

[0002] ADAS (Advanced Driving Assistance System) algorithms usually require a huge amount of data for model training and mid- to late-stage stability verification to continuously improve perception and decision-making capabilities. These data include sensitive information such as faces and license plates collected, which are related to user privacy. In order to protect privacy during data sharing, transmission, and storage, image desensitization has become an indispensable technical means.

[0003] In the existing technology, image desensitization requires the data acquisition equipment to run the desensitization model, but the desensitization model is usually one of the core intellectual property rights of the perception supplier. Therefore, the data acquisition equipment supplier needs to train the desensitization model by itself so that the data acquisition equipment can run the desensitization model. Correspondingly, the processor configuration of the data acquisition equipment needs to be improved. However, improving the processor configuration of the data acquisition equipment will lead to excessively high costs. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides an assisted driving system, an assisted driving control method and a vehicle that support image desensitization, which can not only perform image desensitization processing, but also effectively reduce the cost of data acquisition equipment.

[0005] In a first aspect, an embodiment of the present invention provides an assisted driving system supporting image desensitization, comprising: a camera module, used to perform image acquisition on a target area to obtain first original image data of the target area; an image data copying unit, connected to the camera module, and used to copy the first original image data to obtain first copied image data and second copied image data; an assisted driving unit, connected to the image data copying unit, and used to receive the first copied image data, perform first image processing on the first copied image data to obtain first YUV image data, and perform desensitization area detection on the first YUV image data to obtain desensitized area information; a data acquisition unit, connected to the image data copying unit and the assisted driving unit The driving units are respectively connected to receive the second copied image data from the image data copying unit, perform second image processing on the second copied image data to obtain second YUV image data, and receive the desensitized area information from the auxiliary driving unit, and perform desensitization processing on the second YUV image data using the desensitized area information; wherein the aspect ratio of the second YUV image represented by the second YUV image data is the same as the aspect ratio of the first YUV image represented by the first YUV image data, and the aspect size of the second YUV image represented by the second YUV image data is the same as or in a predetermined ratio to the aspect size of the first YUV image represented by the first YUV image data.

[0006] According to a specific implementation method of an embodiment of the present application, the auxiliary driving unit includes: a first connection module, connected to the image data copying unit, for receiving the first copied image data; a first image processing module, for performing first image processing on the first copied image data to obtain first YUV image data; a desensitized area detection module, for performing desensitized area detection on the first YUV image data through a desensitization model to obtain desensitized area information; or, for detecting an information sensitive area in a first YUV image represented by the first YUV image data, assigning a specified value to the Y value of the information sensitive area to obtain an intermediate YUV image, subtracting the Y value of the intermediate YUV image from that of the first YUV image to obtain a Y value difference map of the first YUV image, and taking the area in the Y value difference map that is not 0 as a desensitized area.

[0007] According to a specific implementation method of an embodiment of the present application, the data acquisition unit includes: a second connection module, connected to the image data copying unit, for receiving the second copied image data; a third connection module, connected to the auxiliary driving unit, for receiving the desensitized area information; a second image processing module, for performing second image processing on the second copied image data to obtain second YUV image data; and a desensitization processing module, for performing desensitization processing on the second YUV image data using the desensitized area information.

[0008] According to a specific implementation method of an embodiment of the present application, the data acquisition unit also includes: an image encoding module, used to encode the second YUV image data after desensitization processing to obtain an image encoding file in a predetermined format; and a coding file saving module, used to save the image coding file to a specified location.

[0009] According to a specific implementation method of an embodiment of the present application, the data acquisition unit also includes: a coding file acquisition module, used to obtain the image coding file from the specified location; an image decoding module, used to decode the image coding file to obtain an image decoding file in a predetermined format; a third image processing module, used to perform a third image processing on the image decoding file to obtain original image data corresponding to the image decoding file, so as to send the original image data to the assisted driving unit.

[0010] According to a specific implementation method of an embodiment of the present application, the first image processing module is also used to perform a third image processing on the quasi-original image data to obtain third YUV image data corresponding to the quasi-original image data; wherein the assisted driving unit also includes: a perception module, which is used to perform perception based on the third YUV image data to obtain a perception result.

[0011] In a second aspect, an embodiment of the present invention further provides an assisted driving control method supporting image desensitization, comprising: performing image acquisition on a target area to obtain first original image data of the target area; copying the first original image data to obtain first copied image data and second copied image data; an assisted driving unit, performing first image processing on the first copied image data to obtain first YUV image data, and performing desensitized area detection on the first YUV image data to obtain desensitized area information; a data acquisition unit, performing second image processing on the second copied image data to obtain second YUV image data, and receiving the desensitized area information from the assisted driving unit, and performing desensitization processing on the second YUV image data using the desensitized area information; wherein the aspect ratio of the second YUV image represented by the second YUV image data is the same as the aspect ratio of the first YUV image represented by the first YUV image data, and the aspect size of the second YUV image represented by the second YUV image data is the same as or in a predetermined ratio to the aspect size of the first YUV image represented by the first YUV image data.

[0012] According to a specific implementation method of an embodiment of the present application, the desensitizing area detection is performed on the first YUV image data to obtain the desensitizing area information, including: performing desensitizing area detection on the first YUV image data through a desensitizing model to obtain the desensitizing area information; or detecting the information sensitive area in the first YUV image represented by the first YUV image data, assigning a specified value to the Y value of the information sensitive area to obtain an intermediate YUV image, subtracting the Y value of the intermediate YUV image from that of the first YUV image to obtain a Y value difference map of the first YUV image, and taking the area in the Y value difference map that is not 0 as a desensitizing area.

[0013] According to a specific implementation method of an embodiment of the present application, the data acquisition unit receives the desensitized area information from the auxiliary driving unit, including: the data acquisition unit receives the desensitized area information from the auxiliary driving unit based on a communication protocol, and desensitizes the second YUV image data according to the desensitized area information.

[0014] According to a specific implementation method of an embodiment of the present application, after the desensitizing area information is desensitized in the data acquisition unit and the second YUV image data is desensitized, the method further includes: encoding the desensitized second YUV image data to obtain an image encoding file in a predetermined format; saving the image encoding file to a specified location; obtaining the image encoding file from the specified location; decoding the image encoding file to obtain an image decoding file in a predetermined format, performing a third image processing on the image decoding file to obtain quasi-original image data corresponding to the image decoding file, and sending the quasi-original image data to the auxiliary driving unit; the auxiliary driving unit performs a third image processing on the quasi-original image data to obtain third YUV image data corresponding to the quasi-original image data; and performing perception based on the third YUV image data to obtain a perception result.

[0015] In a third aspect, an embodiment of the present invention further provides a vehicle, wherein the vehicle is equipped with an assisted driving system, and the assisted driving system is any one of the assisted driving systems provided by the embodiments of the present invention.

[0016] An embodiment of the present invention provides an assisted driving system, assisted driving control method and vehicle supporting image desensitization, wherein a data acquisition unit is respectively connected to an image data copying unit and an assisted driving unit, and is used to receive second copied image data from the image data copying unit, perform second image processing on the second copied image data, obtain second YUV image data, and receive desensitized area information from the assisted driving unit, and perform desensitization processing on the second YUV image data using the desensitized area information. In this way, the data acquisition unit receives the desensitized area information from the assisted driving unit, and performs desensitization processing on the collected image data using the desensitized area information. The data acquisition unit does not need to run the desensitization model by itself, and the processor configuration of the data acquisition unit can be reduced, thereby correspondingly reducing the cost of the data acquisition equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 Schematic diagram of an assisted driving system supporting image desensitization in an embodiment of the present invention; Figure 2 Schematic diagram of an auxiliary driving unit of an auxiliary driving system in an embodiment of the present invention; Figure 3A YUV value distribution diagram of the first YUV image data provided in an embodiment of the present invention; Figure 4 A YUV value distribution diagram of the first YUV image data after desensitization provided by an embodiment of the present invention; Figure 5 A distribution diagram of the difference in YUV values ​​of the first YUV image data before and after desensitization provided by an embodiment of the present invention; Figure 6 Schematic diagram of a data acquisition unit of an auxiliary driving system in an embodiment of the present invention; Figure 7 An assisted driving control method supporting image desensitization is provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0020] It should be clear that the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] As mentioned in the background technology, in the existing solution, data acquisition equipment suppliers need to train the desensitization model themselves. To facilitate the understanding of the present invention, the image desensitization process is briefly described by first taking the L2 level low-cost ADAS image desensitization as an example.

[0022] The L2-level low-cost ADAS front-view integrated machine or domain controller (hereinafter referred to as ADAS device) is limited by the performance limit of the device itself. The mainstream data acquisition solution in the industry usually uses a bypass data acquisition device to collect image data, and transmits the collected images from the camera module to the ADAS device. The images directly output by the camera module have not been desensitized, so the image data desensitization work needs to be performed in the video acquisition and re-injection device (hereinafter referred to as the data acquisition device). The current mainstream data acquisition devices that can perform desensitization use a high-computing system chip combined with a microcontroller chip. An image desensitization model is set in the data acquisition device. The collected image data is operated by the ISP (Image Signal Processor) to obtain a YUV image. The data acquisition device passes the YUV image to its own image desensitization model, obtains a desensitized YUV image after inference, and then encodes the desensitized YUV image to obtain an H264 or H265 file for disk. The data acquisition equipment trains the model itself and the reasoning ability of the model needs to be comparable to that of the perception system chip. Otherwise, the desensitization effect will be significantly different from the desensitization result in the ADAS device. This requires a high configuration of the data acquisition equipment, breaking the balance between data acquisition performance, data desensitization capability and cost.

[0023] Based on this, the researchers found that the new solution applied to image desensitization in assisted driving systems can effectively solve existing technical problems.

[0024] In order to enable those skilled in the art to better understand the technical concept, implementation plan and beneficial effects of the embodiments of the present application, specific embodiments are described in detail below. Embodiment 1

[0025] An embodiment of the present application provides an assisted driving system that supports image desensitization, which can not only perform image desensitization processing, but also reduce the cost of data acquisition equipment.

[0026] Figure 1 Schematic diagram of an assisted driving system supporting image desensitization in an embodiment of the present invention. Figure 1 As shown, the assisted driving system of this embodiment includes: a camera module 1, which is used to perform image acquisition on a target area to obtain first original image data of the target area; an image data copying unit 2, which is connected to the camera module 1, and is used to copy the first original image data to obtain first copied image data and second copied image data; an assisted driving unit 3, which is connected to the image data copying unit 2, and is used to receive the first copied image data, perform first image processing on the first copied image data to obtain first YUV image data, and perform desensitized area detection on the first YUV image data to obtain desensitized area information; a data acquisition unit 4, which is separated from the image data copying unit 2 and the assisted driving unit 3 The auxiliary driving unit 3 is separately connected to the image data copying unit 2, and is used for receiving the second copied image data from the image data copying unit 2, performing second image processing on the second copied image data to obtain second YUV image data, and receiving the desensitized area information from the auxiliary driving unit 3, and performing desensitization processing on the second YUV image data using the desensitized area information; wherein the aspect ratio of the second YUV image represented by the second YUV image data is the same as the aspect ratio of the first YUV image represented by the first YUV image data, and the aspect dimensions of the second YUV image represented by the second YUV image data are the same as or in a predetermined ratio to the aspect dimensions of the first YUV image represented by the first YUV image data.

[0027] The camera module is an important electronic device for image capture. It completes optical imaging through the camera module to realize functions such as photography, information capture and analysis, and visual interaction. The vehicle-mounted camera group includes an optical lens, image sensor, ISP, serializer, and connector. With the continuous iteration of ADAS technical solutions, the number of camera modules has increased from 1V, 6V to 10V, and the module resolution has also been continuously improved. For example, the module resolution has increased from 120W to 200W, and then to 800W, resulting in higher and higher clarity of image data, and the technical capabilities of data acquisition equipment have been continuously raised.

[0028] In an embodiment of the present invention, the camera module is powered on and the device is initialized. After receiving an exposure request, an optical lens captures an image in a target area and projects the image onto an image sensor. After processing, first original image data of the target area is obtained, wherein the target area may be an area of ​​a scene where a vehicle is located in real life. The first original image data may also be referred to as a RAW image.

[0029] The image data copying unit is connected to the camera module, copies the first original image data to obtain first copied image data and second copied image data, and transmits the first copied image data and the second copied image data to the auxiliary driving unit and the data acquisition unit respectively.

[0030] When the auxiliary driving unit begins to enter the normal working process, it sends an exposure request to the camera module. The camera module serializes the first original image data and passes it to the image data replication unit through the communication protocol. In some examples, the communication protocol is GMSL (Gigabit Multimedia Serial Links). The image replication unit replicates the first original image data after serialization to obtain two serialized data, the first replicated image data and the second replicated image data. The auxiliary driving unit is connected to the image data replication unit, and the auxiliary driving unit receives the first replicated image data from the image data replication unit. The first replicated image data is subjected to the first image processing to obtain the first YUV image data, and the first YUV image data is input into the internal desensitization model to perform sensitive area detection to obtain desensitized area information.

[0031] The data acquisition unit is connected to the image data copying unit and the auxiliary driving unit respectively, receives the second copy image data from the image data copying unit, performs the second image processing on the second copy image data, and obtains the second YUV image data. Since the desensitization area information is received from the auxiliary driving unit, the second YUV image data is desensitized corresponding to the desensitization area information. Since the first copy image data and the second copy image data are derived from the same picture, but the image signal processor ISP used for the first image processing and the second image processing is not the same device, the length and width ratio of the first YUV image data and the second YUV image data are the same, and some picture details will be different, for example, picture edge details, picture color, etc. In other words, the length and width of the second YUV image represented by the second YUV image data are the same as or in a predetermined ratio with the length and width of the first YUV image represented by the first YUV image data, then the information of the desensitization area inferred from the first YUV image data in the auxiliary driving unit can be used for the second YUV image data.

[0032] An embodiment of the present invention provides an assisted driving system, assisted driving control method and vehicle supporting image desensitization, wherein a data acquisition unit is respectively connected to an image data copying unit and an assisted driving unit, and is used to receive second copied image data from the image data copying unit, perform second image processing on the second copied image data, obtain second YUV image data, and receive desensitized area information from the assisted driving unit, and perform desensitization processing on the second YUV image data using the desensitized area information. In this way, the data acquisition unit receives the desensitized area information from the assisted driving unit, and performs desensitization processing on the collected image data using the desensitized area information. The data acquisition unit does not need to run the desensitization model by itself, and the processor configuration of the data acquisition unit can be reduced, thereby correspondingly reducing the cost of the data acquisition equipment.

[0033] In some embodiments, the data acquisition unit may use an FPGA (Field-Programmable Gate Array) chip in combination with a microcontroller to interact with the auxiliary driving unit to achieve image desensitization.

[0034] In some embodiments, Figure 2 It is a schematic diagram of the auxiliary driving unit of the auxiliary driving system, such as Figure 2As shown, the auxiliary driving unit 3 includes: a first connection module 31, connected to the image data copying unit 2, for receiving the first copied image data; a first image processing module 32, for performing a first image processing on the first copied image data to obtain first YUV image data; a desensitized area detection module 33, for performing a desensitized area detection on the first YUV image data through a desensitization model to obtain desensitized area information; or, for detecting an information sensitive area in the first YUV image represented by the first YUV image data, assigning a specified value to the Y value of the information sensitive area to obtain an intermediate YUV image, subtracting the Y value of the intermediate YUV image from that of the first YUV image to obtain a Y value difference map of the first YUV image, and taking the area in the Y value difference map that is not 0 as a desensitized area.

[0035] The auxiliary driving unit 3 includes a first connection module 31, a first image processing module 32 and a desensitized area detection module 33, which are interconnected and work together to obtain the desensitized information of the first original image data.

[0036] One end of the first connection module 31 is connected to the image copying unit 2, and the other end is connected to the first image processing module 32, and receives the first copy image data from the image copying unit 2 and transmits it to the first image processing module 32. After the auxiliary driving unit 3 is powered on, the device is initialized, and the first image processing module 32 performs the first image processing on the first copy image data to obtain the first YUV image data. In some examples, the first image processing module 32 includes an ISP, and the first copy image data is a RAW image, which is converted into an image signal output in a format such as RGB, YUV, etc. by the image signal processor ISP, and the RAW image that cannot be directly analyzed is converted into an image that can be analyzed.

[0037] The first YUV image data is input to the desensitization area detection module 33, and the first YUV image data is subjected to desensitization area detection through the desensitization model. After the desensitization model detects the sensitive area information (such as face or license plate, etc.) in the first YUV image data, the sensitive area detection result (the position and size of the face and license plate in the first YUV image data) is serialized and packaged into a lightweight data exchange format (such as Json format) or other serializable network sequence data BagDATA. For example, other serializable network sequence data BagDATA can be {"frameid":1, "type": "face", "x":137, "y":855, "w":12, "h":57}.

[0038] The sensitive area detection result can be obtained directly by the auxiliary driving unit 3 through the desensitization model, or by the auxiliary driving unit 3 performing difference calculation on the pixel values ​​of the first YUV image data before desensitization and the first YUV image data after desensitization. For example: assuming that the first YUV image data is Y1, and the first YUV image data after desensitization is Y1c, Y1 is a 4×4 pixel YUV image, and the YUV value is as follows: Figure 3 As shown, the sensitive area is in the center of the picture, the YUV value is blurred into 127 values, and the obtained Y1c is as follows Figure 4 As shown, the YUV values ​​of the two pictures are subtracted to obtain Figure 5 In the difference graph, the part that is not equal to 0 is the desensitization result.

[0039] In some embodiments, Figure 6 It is a schematic diagram of the data acquisition unit of the auxiliary driving system, such as Figure 6 As shown, the data acquisition unit 4 includes: a second connection module 41, connected to the image data copying unit 2, for receiving the second copied image data; a third connection module 42, connected to the auxiliary driving unit 3, for receiving the desensitized area information; a second image processing module 43, for performing second image processing on the second copied image data to obtain second YUV image data; a desensitization processing module 44, for performing desensitization processing on the second YUV image data using the desensitized area information.

[0040] The data acquisition unit 4 includes a second connection module 41, a third connection module 42, a second image processing module 43 and a desensitization processing module 44, which are interconnected and work together to perform desensitization processing on the first original image data.

[0041] One end of the second connection module 41 is connected to the image copying unit 2, and the other end is connected to the second image processing module 43, receiving the second copy image data from the image copying unit 2, and transmitting it to the second image processing module 43. The second image processing module 43 performs second image processing on the second copy image data to obtain second YUV image data. In some examples, the second image processing module 43 includes a class ISP, which is different from the ISP of the first image processing module 32. The second copy image data is a RAW image, which is converted into an image signal in a format such as RGB, YUV, etc. by an image signal processor class ISP for output.

[0042] The third connection module 42 is connected to the desensitization processing module 44 at one end and to the auxiliary driving unit 3 at the other end. After power-on, the communication middleware in the data acquisition unit 4 establishes a TCP (Transmission Control Protocol) connection with the same-star server of the auxiliary driving unit 3 as a client, and then maintains a heartbeat for a fixed time to confirm the working status of both parties, so that the data acquisition unit 4 and the auxiliary driving unit 3 interact, and receive the desensitized area information of the auxiliary driving unit 3 through the communication protocol. In some examples, the communication protocol can interact, such as CAN (Controller Area Network) communication protocol bus, Ethernet, etc. After obtaining the desensitization result, the auxiliary driving unit 3 sends BagDATA to the desensitization processing module 44 in the data acquisition unit 4 through the custom TCP protocol.

[0043] The second YUV image data and the frame identification information corresponding to the second YUV image data are passed to the video data desensitization processing module 44 of the data acquisition unit 4. The desensitization processing module 44 puts the data information into the data queue to be desensitized. After receiving the desensitized area information BagDATA from the auxiliary driving unit 3, the data acquisition unit 4 parses the content to obtain the frame identification information, and uses the frame identification information to retrieve the corresponding second YUV image data in the data queue. After obtaining the corresponding second YUV image data picture, since the data acquisition unit 4 in some embodiments of the present invention uses FPGA, the data acquisition unit 4 can use its internal FPGA module to perform Gaussian or other fuzzy processing on the area position corresponding to the data acquisition unit 4 that needs to be desensitized, and obtain the desensitized second YUV image data after processing.

[0044] In some embodiments, the data acquisition unit further includes: an image encoding module, used to encode the second YUV image data after desensitization processing to obtain an image encoding file in a predetermined format; and a coding file saving module, used to save the image encoding file to a specified location.

[0045] The data acquisition unit includes an image encoding module and an encoding file saving module. After the image data is desensitized, the image data needs to be encoded and saved. In the image encoding module, the second YUV image data after desensitization is transmitted to the internal video encoder for encoding. In the encoding file saving module, the encoded image encoding file is saved to a specified location (in a database or other storage device). In some examples, the image encoding file can be an h264 or h265 code stream file, which is a file format processed by an efficient intra-frame and inter-frame compression algorithm.

[0046] In some embodiments, the data acquisition unit further includes: a coding file acquisition module, used to obtain the image coding file from the designated location; an image decoding module, used to decode the image coding file to obtain an image decoding file in a predetermined format; a third image processing module, used to perform a third image processing on the image decoding file to obtain original image data corresponding to the image decoding file, so as to send the original image data to the assisted driving unit.

[0047] After the image encoding file is saved, a decoding operation should also be performed. After power-on, the data acquisition unit is operated in the recharge mode, and the client in the data acquisition unit is not actively connected to the auxiliary driving unit server.

[0048] The data acquisition unit includes a coding file acquisition module, an image decoding module and a third image processing module. In the coding file acquisition module, an image coding file (h264 or h265 code stream file) is loaded from a specified location (in a database or other storage device). In the image decoding module, the image coding file is decoded into a second YUV image data that has been desensitized by a decoder to obtain an image decoding file in a predetermined format. In the third image processing module, the image decoding file is input into an image signal processor that is matched with the second image processing module of the data acquisition unit, and the third image processing is performed to process the image decoding file into quasi-raw image data consistent with the format of the camera module, and the processed quasi-raw image data is serialized and passed to the auxiliary driving unit through the communication protocol GMSL.

[0049] The third image processing is the inverse process of the second image processing. The second image processing ISP is to expose the camera module photosensitive sensor to obtain the first original image data (RAW image) in a predetermined format (such as Bayer format), and obtain the specific YUV value of each point through the red, blue and green values ​​of the photosensitive sensor in combination with the sampling algorithm to obtain the second YUV image data. Therefore, if the sampling algorithm is known in the third image processing, each pixel point of the second YUV image data can be used to reversely calculate the Bayer format value corresponding to the pixel point, and obtain a RAW image similar to the original image data.

[0050] In some embodiments, the first image processing module is further used to perform a third image processing on the quasi-original image data to obtain third YUV image data corresponding to the quasi-original image data; wherein the assisted driving unit further includes: a perception module, used to perform perception based on the third YUV image data to obtain a perception result.

[0051] The auxiliary driving unit also includes a perception module. After the quasi-original image data is transmitted to the auxiliary driving unit, the quasi-original image data is subjected to a third image processing in the first image processing module to obtain third YUV image data corresponding to the quasi-original image data, and the third YUV image data is transmitted to the perception module to obtain a perception result.

[0052] The perception model of the perception module is equivalent to the large model algorithm for running image recognition. In some examples, the perception model can be based on two strategies, one is rule-based and the other is based on the AI ​​large model. The rule-based strategy requires grayscale processing of the third YUV image data, and convolution matching of the target attribute based on the edge information. For example, if the third YUV image data is a license plate, the matching result is the license plate, and the position of the license plate in the image is output; the AI ​​large model requires elephant image input for training, continuous iteration of model parameters, manual annotation of regional location categories, third YUV image data input, and output of the corresponding position to obtain the result.

[0053] The assisted driving unit provides the perception results to downstream operations (such as fusion algorithms, regulatory control algorithms, etc.) to achieve the purpose of feedback to the data acquisition unit.

[0054] The desensitization result of the third YUV image data is highly consistent with the desensitization result of the auxiliary driving unit. The desensitization action of the data acquisition unit uses the desensitization information given by the desensitization model running in the auxiliary driving unit. Therefore, the desensitization result is very similar to the desensitization result obtained by the perception model during acquisition, and has the least impact on the perception result of the perception model. Therefore, when this part of the data is re-injected into the perception model of the auxiliary driving unit on the re-injection test bench, the reasoning result of the perception model is very similar to the perception result of the perception model of the auxiliary driving unit at the time of acquisition, which is conducive to reproducing the perception problem. Embodiment 2

[0055] An embodiment of the present application provides an assisted driving control method supporting image desensitization, which can not only perform image desensitization processing, but also effectively reduce the cost of data acquisition equipment. The assisted driving control method supporting image desensitization of this embodiment can be applied to the assisted driving control system supporting image desensitization described in the aforementioned embodiment.

[0056] Figure 7 A flowchart of an assisted driving control method supporting image desensitization provided in an embodiment of the present application is shown in FIG. Figure 7 As shown, an assisted driving control method supporting image desensitization in this embodiment includes: S11, performing image acquisition on the target area to obtain first original image data of the target area; In an embodiment of the present invention, the camera module is powered on and the device is initialized. After receiving an exposure request, an optical lens captures an image in a target area and projects the image onto an image sensor. After processing, first original image data of the target area is obtained, wherein the target area may be an area of ​​a scene where a vehicle is located in real life, and the first original image data may be a RAW image.

[0057] S12, copying the first original image data to obtain first copied image data and second copied image data; The first original image data is copied to obtain first copied image data and second copied image data, and the first copied image data and the second copied image data are respectively transmitted to relevant modules.

[0058] S13, the auxiliary driving unit performs first image processing on the first copied image data to obtain first YUV image data, and performs desensitization area detection on the first YUV image data to obtain desensitization area information; When the auxiliary driving unit begins to enter the normal working process, it sends an exposure request to the camera module. The camera module serializes the first original image data and passes it to the image data copying unit through the communication protocol. In some examples, the communication protocol is GMSL. The image copying unit copies the first original image data after serialization to obtain two serialized data, the first copied image data and the second copied image data. The auxiliary driving unit is connected to the image data copying unit, and the auxiliary driving unit receives the first copied image data from the image data copying unit. Perform the first image processing on the first copied image data to obtain the first YUV image data, and input the first YUV image data into the internal desensitization model to detect the sensitive area and obtain the desensitized area information.

[0059] S14, a data acquisition unit, performs second image processing on the second copied image data to obtain second YUV image data, receives the desensitized area information from the auxiliary driving unit, and uses the desensitized area information to perform desensitization processing on the second YUV image data; wherein, the aspect ratio of the second YUV image represented by the second YUV image data is the same as the aspect ratio of the first YUV image represented by the first YUV image data, and the aspect dimensions of the second YUV image represented by the second YUV image data are the same as or in a predetermined ratio to the aspect dimensions of the first YUV image represented by the first YUV image data.

[0060] The data acquisition unit is connected to the image data copying unit and the auxiliary driving unit respectively, receives the second copied image data from the image data copying unit, performs the second image processing on the second copied image data, and obtains the second YUV image data. Since the desensitized area information is received from the auxiliary driving unit, the second YUV image data is desensitized corresponding to the desensitized area information. Since the first copied image data and the second copied image data are derived from the same picture, but the first image processing and the second image processing are not performed on the same device, the length and width ratio of the first YUV image data and the second YUV image data are the same, and some picture details will be different, for example, picture edge details, picture color, etc. In other words, the length and width of the second YUV image represented by the second YUV image data are the same as or in a predetermined ratio to the length and width of the first YUV image represented by the first YUV image data, then the information of the desensitized area inferred from the first YUV image data in the auxiliary driving unit can be used for the second YUV image data.

[0061] An embodiment of the present invention provides an assisted driving control method supporting image desensitization, comprising: a data acquisition unit, performing a second image processing on a second copied image data to obtain a second YUV image data, and receiving desensitized area information from an assisted driving unit, and performing desensitization processing on the second YUV image data using the desensitized area information. In this way, the data acquisition unit receives the desensitized area information from the assisted driving unit, and performs desensitization processing on the collected image data using the desensitized area information. The data acquisition unit does not need to run the desensitization model itself, and the processor configuration of the data acquisition device can be reduced, which correspondingly reduces the cost of the data acquisition unit.

[0062] In some embodiments, the performing desensitized area detection on the first YUV image data to obtain desensitized area information includes: performing desensitized area detection on the first YUV image data through a desensitization model to obtain desensitized area information; or detecting the information sensitive area in the first YUV image represented by the first YUV image data, assigning a specified value to the Y value of the information sensitive area to obtain an intermediate YUV image, subtracting the Y value of the intermediate YUV image from that of the first YUV image to obtain a Y value difference map of the first YUV image, and taking the area in the Y value difference map that is not 0 as a desensitized area.

[0063] The first YUV image data is input into the desensitization area detection module, and the desensitization area detection is performed on the first YUV image data through the desensitization model. After the desensitization model detects the sensitive area information (such as face or license plate, etc.) in the first YUV image data, the sensitive area detection result (the position and size of the face and license plate in the first YUV image data) is serialized and packaged into a lightweight data exchange format (such as Json format) or other serializable network sequence data BagDATA. For example, other serializable network sequence data BagDATA can be {"frameid":1, "type": "face", "x":137, "y":855, "w":12, "h":57}.

[0064] The sensitive area detection result can be obtained directly by the auxiliary driving unit through the desensitization model, or by the auxiliary driving unit performing difference calculation on the pixel values ​​of the first YUV image data before desensitization and the first YUV image data after desensitization. For example: assuming that the first YUV image data is Y1, and the first YUV image data after desensitization is Y1c, Y1 is a 4×4 pixel YUV, and the Y value is Figure 3 As shown, the sensitive area is in the center of the image, the Y value is blurred to 127, and the obtained Y1c is as follows Figure 4 As shown, the Y value of the two pictures is subtracted to get Figure 5 In the difference graph, the part that is not equal to 0 is the desensitization result.

[0065] In some embodiments, the data acquisition unit receives the desensitized area information from the auxiliary driving unit, including: the data acquisition unit receives the desensitized area information from the auxiliary driving unit based on a communication protocol, and desensitizes the second YUV image data according to the desensitized area information.

[0066] The data acquisition unit is connected to the auxiliary driving unit through a wire. After power-on, the communication middleware in the data acquisition unit establishes a TCP (Transmission Control Protocol) connection with the same-star server of the auxiliary driving unit as a client, and then maintains a fixed heartbeat to confirm the working status of both parties, so that the data acquisition unit and the auxiliary driving unit can interact and receive the desensitized area information of the auxiliary driving unit through the communication protocol. In some examples, the communication protocol can interact, such as CAN (Controller Area Network) communication protocol bus, Ethernet, etc. After obtaining the desensitization result, the auxiliary driving unit sends BagDATA to the data acquisition unit through a custom TCP protocol.

[0067] During the data acquisition process, the data acquisition unit directly obtains the desensitized information given by the perception results in the auxiliary driving unit through the interactive protocol, and uses this desensitized information to perform fuzzy desensitization processing on the video data. In some examples, the fuzzy desensitization processing can be performed using an encoder, which is efficient and low-occupancy, and the processor configuration of the data acquisition equipment can be reduced.

[0068] In some embodiments, after the data acquisition unit desensitizes the second YUV image data according to the desensitized area information, the method further includes: encoding the desensitized second YUV image data to obtain an image encoding file in a predetermined format; saving the image encoding file to a specified location; obtaining the image encoding file from the specified location; decoding the image encoding file to obtain an image decoding file in a predetermined format, performing a third image processing on the image decoding file to obtain quasi-original image data corresponding to the image decoding file, and sending the quasi-original image data to the auxiliary driving unit; the auxiliary driving unit performs a third image processing on the quasi-original image data to obtain third YUV image data corresponding to the quasi-original image data; and performing perception based on the third YUV image data to obtain a perception result.

[0069] After desensitizing the second YUV image data, the image data needs to be encoded and saved. The desensitized second YUV image data is transmitted to the internal video encoder for encoding, and the encoded image file is saved to a specified location (in a database or other storage device). In some examples, the image encoding file can be an h264 or h265 code stream file, which is a file format processed by an efficient intra-frame and inter-frame compression algorithm.

[0070] After the image encoding file is saved, a decoding operation should also be performed. After power-on, the data acquisition unit is operated in the recharge mode, and the client in the data acquisition unit is not actively connected to the auxiliary driving unit server.

[0071] Load the image encoding file (h264 or h265 code stream file) from the specified location (in the database or other storage device), decode the image encoding file into the desensitized second YUV image data through the decoder, and obtain the image decoding file in the predetermined format. Input the image decoding file into the image signal processor, perform the third image processing, process the image decoding file into the same format as the camera module, and transmit the processed quasi-raw image data to the auxiliary driving unit through the communication protocol GMSL after serialization.

[0072] The third image processing is the inverse process of the second image processing. The second image processing ISP is to expose the camera module photosensitive sensor to obtain the first original image data (RAW image) in a predetermined format (such as Bayer format), and obtain the specific YUV value of each point through the red, blue and green values ​​of the photosensitive sensor in combination with the sampling algorithm to obtain the second YUV image data. Therefore, if the sampling algorithm is known in the third image processing, each pixel point of the second YUV image data can be used to reversely calculate the Bayer format value corresponding to the pixel point, and obtain a RAW image similar to the original image data.

[0073] After the quasi-original image data is transmitted to the auxiliary driving unit, the quasi-original image data is subjected to third image processing in the first image processing module to obtain third YUV image data corresponding to the quasi-original image data, and the third YUV image data is sensed to obtain a sensed result.

[0074] The perception model used to perceive the third YUV image data is equivalent to the large model algorithm for running image recognition. In some examples, the perception model can be based on two strategies, one is rule-based and the other is based on the AI ​​large model. The rule-based strategy requires grayscale processing of the third YUV image data, and convolution matching of the target attribute based on the edge information. For example, if the third YUV image data is a license plate, the matching result is the license plate, and the position of the license plate in the image is output; the AI ​​large model requires elephant image input for training, continuous iteration of model parameters, manual annotation of regional location categories, third YUV image data input, and output of the corresponding position to obtain the result.

[0075] Taking the above operation process as an example, the data acquisition unit and the auxiliary driving unit equipped with this system encounter a passenger car V1 with a license plate in front and a pedestrian P1 driving in the opposite direction of my vehicle during the data acquisition process. During the operation of the whole system, the data acquisition unit obtains the second YUV image data Y2 at this moment, and the auxiliary driving unit obtains the first YUV image data Y1 at this moment, and obtains the desensitized information in Y1. In the desensitized information, the coordinate position of P1's face in the picture is 1168, 1287, 1195, 1271 (top, right, bottom, left, the following parameters are the same), and the coordinate position of V1's license plate in the picture is 1213, 2203, 1226, 2174. The assisted driving unit transmits the two sets of desensitized information to the data acquisition unit through TCP. The data acquisition unit performs Gaussian or other fuzzy processing operations on the YUV values ​​of the corresponding sensitive positions in the current image Y2 to obtain the desensitized Y2c, and then transmits Y2c to the internal video encoder for encoding. The encoded image encoding file is saved to the specified location (database or other storage device). The video data stored at this time is the desensitized data.

[0076] When it is necessary to re-inject the scene for problem reproduction or performance analysis, the data acquisition unit switches to the re-injection mode, loads the image encoding file (h264 or h265 code stream file) from the specified location (database or other storage device), and decodes the image encoding file into a second YUV image data that has been desensitized through a decoder to obtain an image decoding file in a predetermined format. After decoding, Y2c is obtained. At this time, because the sensitive area in the image has been Gaussian or wiped, the information cannot be recovered. At the same time, because the desensitized area of ​​the image is the algorithm provided by the auxiliary driving unit itself, the negative impact of the desensitized image on the perception model when reasoning can be minimized. Embodiment 3

[0077] An embodiment of the present invention further provides a vehicle, on which is configured an assisted driving system, and the assisted driving system is any one of the assisted driving systems provided in the aforementioned embodiments.

[0078] An embodiment of the present invention provides a vehicle, which is equipped with an assisted driving system, and the assisted driving system includes: a data acquisition unit, which is respectively connected to an image data copying unit and an assisted driving unit, and is used to receive second copied image data from the image data copying unit, perform second image processing on the second copied image data, and obtain second YUV image data, and receive desensitized area information from the assisted driving unit, and perform desensitization processing on the second YUV image data using the desensitized area information. In this way, the data acquisition unit receives the desensitized area information from the assisted driving unit, and performs desensitization processing on the collected image data using the desensitized area information. The data acquisition unit does not need to run the desensitization model by itself, and the processor configuration of the data acquisition unit can be reduced, which correspondingly reduces the cost of the data acquisition equipment.

[0079] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0080] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0081] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. An assisted driving system supporting image desensitization, characterized in that: include: The camera module is used to capture an image of the target area to obtain first original image data of the target area; An image data copying unit, connected to the camera module, and configured to copy the first original image data to obtain first copy image data and second copy image data; an auxiliary driving unit, connected to the image data copying unit, and configured to receive the first copied image data, perform first image processing on the first copied image data to obtain first YUV image data, and perform desensitized area detection on the first YUV image data to obtain desensitized area information; A data acquisition unit is connected to the image data copying unit and the auxiliary driving unit respectively, and is used to receive the second copied image data from the image data copying unit, perform second image processing on the second copied image data to obtain second YUV image data, and receive the desensitized area information from the auxiliary driving unit, and perform desensitization processing on the second YUV image data using the desensitized area information; wherein the aspect ratio of the second YUV image represented by the second YUV image data is the same as the aspect ratio of the first YUV image represented by the first YUV image data, and the aspect dimensions of the second YUV image represented by the second YUV image data are the same as or in a predetermined ratio to the aspect dimensions of the first YUV image represented by the first YUV image data.

2. The system according to claim 1, characterized in that The auxiliary driving unit comprises: A first connection module, connected to the image data copying unit, and configured to receive the first copied image data; A first image processing module, used for performing first image processing on the first copied image data to obtain first YUV image data; A desensitized area detection module is used to perform desensitized area detection on the first YUV image data through a desensitization model to obtain desensitized area information; or, to detect the information sensitive area in the first YUV image represented by the first YUV image data, assign a specified value to the Y value of the information sensitive area, obtain an intermediate YUV image, subtract the Y value of the intermediate YUV image from that of the first YUV image, obtain a Y value difference map of the first YUV image, and use the area in the Y value difference map that is not 0 as a desensitized area.

3. The system according to claim 1, characterized in that The data acquisition unit comprises: A second connection module, connected to the image data copying unit, and configured to receive the second copied image data; A third connection module, connected to the auxiliary driving unit, and configured to receive the desensitized area information; A second image processing module, used for performing second image processing on the second copied image data to obtain second YUV image data; A desensitization processing module is used to perform desensitization processing on the second YUV image data using the desensitization area information.

4. The system according to claim 3, characterized in that The data acquisition unit further includes: An image encoding module, used for encoding the second YUV image data after the desensitization process to obtain an image encoding file in a predetermined format; The coding file saving module is used to save the image coding file to a specified location.

5. The system according to claim 4, characterized in that The data acquisition unit further includes: A coding file acquisition module, used for acquiring the image coding file from the specified location; An image decoding module, used for decoding the image encoding file to obtain an image decoding file in a predetermined format; The third image processing module is used to perform third image processing on the image decoding file to obtain the original image data corresponding to the image decoding file, so as to send the original image data to the auxiliary driving unit.

6. The system according to claim 5, characterized in that The first image processing module is further used to perform a third image processing on the quasi-original image data to obtain third YUV image data corresponding to the quasi-original image data; Wherein, the auxiliary driving unit further includes: A perception module is used to perform perception based on the third YUV image data to obtain a perception result.

7. An assisted driving control method supporting image desensitization, characterized in that: include: Capturing an image of the target area to obtain first original image data of the target area; Copying the first original image data to obtain first copied image data and second copied image data; The auxiliary driving unit performs first image processing on the first copied image data to obtain first YUV image data, and performs desensitized area detection on the first YUV image data to obtain desensitized area information; The data acquisition unit performs a second image processing on the second copied image data to obtain second YUV image data, receives the desensitized area information from the auxiliary driving unit, and performs desensitization processing on the second YUV image data using the desensitized area information; wherein the aspect ratio of the second YUV image represented by the second YUV image data is the same as the aspect ratio of the first YUV image represented by the first YUV image data, and the aspect dimensions of the second YUV image represented by the second YUV image data are the same as or in a predetermined ratio to the aspect dimensions of the first YUV image represented by the first YUV image data.

8. The method according to claim 7, characterized in that The performing desensitization area detection on the first YUV image data to obtain desensitization area information includes: Perform desensitization area detection on the first YUV image data by using a desensitization model to obtain desensitization area information; or Detect the information sensitive area in the first YUV image represented by the first YUV image data, assign a specified value to the Y value of the information sensitive area to obtain an intermediate YUV image, subtract the Y value of the intermediate YUV image from that of the first YUV image to obtain a Y value difference map of the first YUV image, and use the area in the Y value difference map that is not 0 as a desensitized area.

9. The method according to claim 7, characterized in that: The data acquisition unit receives the desensitized area information from the auxiliary driving unit, including: The data acquisition unit receives the desensitized area information from the auxiliary driving unit based on a communication protocol, and desensitizes the second YUV image data according to the desensitized area information.

10. The method according to claim 7, characterized in that After the desensitizing area information is desensitized by the data acquisition unit and the second YUV image data is desensitized, the method further includes: Encoding the second YUV image data after the desensitization process to obtain an image encoding file in a predetermined format; Saving the image encoding file to a specified location; Acquire the image encoding file from the specified location; Decoding the image encoding file to obtain an image decoding file in a predetermined format, performing a third image processing on the image decoding file to obtain quasi-original image data corresponding to the image decoding file, and sending the quasi-original image data to the auxiliary driving unit; The auxiliary driving unit performs a third image processing on the quasi-original image data to obtain third YUV image data corresponding to the quasi-original image data; and performs perception based on the third YUV image data to obtain a perception result.

11. A vehicle equipped with an auxiliary driving system, characterized in that: The assisted driving system is the assisted driving system described in any one of claims 1 to 6.

Images