Assisted driving system supporting image desensitization, assisted driving control method, and vehicle

Abstract

The embodiment of the application discloses an auxiliary driving system supporting image desensitization, an auxiliary driving control method and a vehicle, relates to the technical field of auxiliary driving, and can not only perform image desensitization processing, but also effectively reduce the cost of data acquisition equipment. The system comprises a camera module, which is used for obtaining first original image data; an image data replication unit, which is used for obtaining first replication image data and second replication image data from the camera module; an auxiliary driving unit connected with the image data replication unit, which is used for performing first image processing on the first replication image data to obtain first YUV image data, and performing desensitization on the first YUV image data to obtain desensitization area information; and a data acquisition unit connected with the image data replication unit and the auxiliary driving unit, which is used for performing second image processing on the second replication image data to obtain second YUV image data, and performing desensitization processing on the second YUV image data by using the desensitization area information. The application is suitable for image desensitization operation of the auxiliary driving system.

Description

Technical Field

[0001] This invention relates to the field of driver assistance technology, and in particular to a driver assistance system, driver assistance control method and vehicle that support image desensitization. Background Technology

[0002] ADAS (Advanced Driving Assistance System) algorithms typically require massive amounts of data for model training and mid-to-late-stage stability verification to continuously improve perception and decision-making capabilities. This data includes sensitive information related to user privacy, such as collected facial and license plate data. To protect privacy during data sharing, transmission, and storage, image anonymization has become an indispensable technical means.

[0003] In existing technologies, image desensitization requires data acquisition equipment to run a desensitization model. However, the desensitization model is usually one of the core intellectual property rights of the perception supplier. Therefore, the data acquisition equipment supplier needs to train the desensitization model themselves so that the data acquisition equipment can run the desensitization model. Accordingly, the processor configuration of the data acquisition equipment needs to be improved. However, improving the processor configuration of the data acquisition equipment will lead to excessively high costs. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide an assisted driving system, assisted driving control method and vehicle that support image desensitization, which can perform image desensitization processing and effectively reduce the cost of data acquisition equipment.

[0005] In a first aspect, embodiments of the present invention provide an assisted driving system supporting image desensitization, comprising: a camera module for acquiring images of a target area to obtain first original image data of the target area; an image data copying unit connected to the camera module for copying the first original image data to obtain first copied image data and second copied image data; an assisted driving unit connected to the image data copying unit for receiving the first copied image data, performing first image processing on the first copied image data to obtain first YUV image data, and performing desensitization region detection on the first YUV image data to obtain desensitization region information; and a data acquisition unit connected to the image data copying unit and the assisted driving unit. The driving units are connected to each other and are used to receive the second copied image data from the image data copying unit, perform second image processing on the second copied image data to obtain second YUV image data, and receive the desensitized area information from the assisted driving unit, and perform desensitization processing on the second YUV image data using the desensitized area information; wherein, the aspect ratio of the second YUV image represented by the second YUV image data is the same as the aspect ratio of the first YUV image represented by the first YUV image data, and the length and width dimensions of the second YUV image represented by the second YUV image data are the same as or in a predetermined ratio to the length and width dimensions of the first YUV image represented by the first YUV image data.

[0006] According to a specific implementation of an embodiment of this application, the assisted driving unit includes: a first connection module connected to the image data copying unit, used to receive the first copied image data; a first image processing module, used to perform first image processing on the first copied image data to obtain first YUV image data; a desensitized region detection module, used to perform desensitized region detection on the first YUV image data through a desensitization model to obtain desensitized region information; or, used to detect information-sensitive regions in the first YUV image represented by the first YUV image data, assign a specified value to the Y value of the information-sensitive region to obtain an intermediate YUV image, subtract the Y value of the intermediate YUV image from the Y value of the first YUV image to obtain a Y value difference map of the first YUV image, and take the non-zero region in the Y value difference map as the desensitized region.

[0007] According to a specific implementation of an embodiment of this application, the data acquisition unit includes: a second connection module connected to the image data copying unit for receiving the second copied image data; a third connection module connected to the assisted driving unit for receiving the desensitized area information; a second image processing module for performing second image processing on the second copied image data to obtain second YUV image data; and a desensitization processing module for using the desensitized area information to perform desensitization processing on the second YUV image data.

[0008] According to a specific implementation of an embodiment of this application, the data acquisition unit further includes: an image encoding module, used to encode the desensitized second YUV image data to obtain an image encoding file in a predetermined format; and an encoding file saving module, used to save the image encoding file to a specified location.

[0009] According to a specific implementation of an embodiment of this application, the data acquisition unit further includes: an encoding file acquisition module, used to acquire the image encoding file from the specified location; an image decoding module, used to decode the image encoding file to obtain an image decoding file of a predetermined format; and a third image processing module, used to perform third image processing on the image decoding file to obtain a class of original image data corresponding to the image decoding file, so as to send the class of original image data to the assisted driving unit.

[0010] According to a specific implementation of an embodiment of this application, the third image processing module is further configured to perform third image processing on the class of original image data to obtain third YUV image data corresponding to the class of original image data; wherein, the assisted driving unit further includes: a perception module, configured to perform perception based on the third YUV image data to obtain perception results.

[0011] Secondly, embodiments of the present invention also provide an assisted driving control method supporting image desensitization, comprising: acquiring images of a target area to obtain first original image data of the target area; copying the first original image data to obtain first copied image data and second copied image data; an assisted driving unit performing first image processing on the first copied image data to obtain first YUV image data, and performing desensitization region detection on the first YUV image data to obtain desensitization region information; a data acquisition unit performing second image processing on the second copied image data to obtain second YUV image data, and receiving the desensitization region information from the assisted driving unit, and using the desensitization region information to perform desensitization processing on the second YUV image data; wherein, the aspect ratio of the second YUV image represented by the second YUV image data is the same as the aspect ratio of the first YUV image represented by the first YUV image data, and the length and width dimensions of the second YUV image represented by the second YUV image data are the same as or at a predetermined ratio to the length and width dimensions of the first YUV image represented by the first YUV image data.

[0012] According to a specific implementation of an embodiment of this application, the step of detecting desensitized regions in the first YUV image data to obtain desensitized region information includes: detecting desensitized regions in the first YUV image data using a desensitization model to obtain desensitized region information; or detecting information-sensitive regions in the first YUV image represented by the first YUV image data, assigning a specified value to the Y value of the information-sensitive region to obtain an intermediate YUV image, subtracting the Y value of the intermediate YUV image from that of the first YUV image to obtain a Y value difference map of the first YUV image, and using the non-zero regions in the Y value difference map as desensitized regions.

[0013] According to a specific implementation of an embodiment of this application, the data acquisition unit receives the desensitized region information from the driver assistance unit, including: the data acquisition unit receives the desensitized region information from the driver assistance unit based on a communication protocol, and performs desensitization processing on the second YUV image data according to the desensitized region information.

[0014] According to a specific implementation of this application, after the data acquisition unit desensitizes the second YUV image data using the desensitized region information, the method further includes: encoding the desensitized second YUV image data to obtain an image encoding file of a predetermined format; saving the image encoding file to a designated location; retrieving the image encoding file from the designated location; decoding the image encoding file to obtain an image decoding file of a predetermined format; performing third image processing on the image decoding file to obtain near-original image data corresponding to the image decoding file; sending the near-original image data to the assisted driving unit; the assisted driving unit performing third image processing on the near-original image data to obtain third YUV image data corresponding to the near-original image data; and performing perception based on the third YUV image data to obtain a perception result.

[0015] Thirdly, embodiments of the present invention also provide a vehicle equipped with an assisted driving system, wherein the assisted driving system is any of the assisted driving systems provided in embodiments of the present invention.

[0016] The embodiments of the present invention provide an assisted driving system, assisted driving control method, and vehicle supporting image desensitization. A data acquisition unit is connected to an image data copying unit and an assisted driving unit, respectively. The data acquisition unit receives second copied image data from the image data copying unit, performs second image processing on the second copied image data to obtain second YUV image data, and receives desensitized region information from the assisted driving unit, using the desensitized region information to perform desensitization processing on the second YUV image data. In this way, the data acquisition unit receives desensitized region information from the assisted driving unit and uses the desensitized region information to perform desensitization processing on the acquired image data. This eliminates the need for the data acquisition unit to run its own desensitization model, reducing the processor configuration required by the data acquisition unit and correspondingly lowering the cost of the data acquisition equipment. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of an assisted driving system supporting image desensitization in an embodiment of the present invention;

[0019] Figure 2 This is a schematic diagram of the driver assistance unit of the driver assistance system in an embodiment of the present invention;

[0020] Figure 3 This is a YUV value distribution map of the first YUV image data provided in an embodiment of the present invention;

[0021] Figure 4 A distribution map of YUV values ​​in the first desensitized YUV image data provided for an embodiment of the present invention;

[0022] Figure 5 A distribution map of the difference in YUV values ​​of the first YUV image data before and after desensitization is provided for an embodiment of the present invention;

[0023] Figure 6 This is a schematic diagram of the data acquisition unit of the driver assistance system in an embodiment of the present invention;

[0024] Figure 7 An assisted driving control method supporting image desensitization is provided as an embodiment of the present invention. Detailed Implementation

[0025] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0026] It should be understood that the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0027] As mentioned in the background section, in existing solutions, data acquisition equipment suppliers need to train their own desensitization models. To facilitate understanding of this invention, we will first take low-cost ADAS image desensitization at the L2 level as an example to briefly explain the image desensitization process.

[0028] Low-cost L2-level ADAS front-view all-in-one machines or domain controllers (hereinafter referred to as ADAS devices) are limited by their own performance limits. The mainstream data acquisition solutions in the industry typically use bypass data acquisition devices to acquire image data, and then transmit the acquired images to the ADAS device via a camera module. The images directly output from the camera module are not image-sensitized; therefore, image data desensitization needs to be performed in the video acquisition and feedback device (hereinafter referred to as the data acquisition device). Currently, mainstream data acquisition devices capable of desensitization use high-performance system chips combined with microcontroller chips. An image desensitization model is set up within the data acquisition device. After processing the acquired image data using an ISP (Image Signal Processor), a YUV image is obtained. The data acquisition device then passes the YUV image to its own image desensitization model for inference, resulting in a desensitized YUV image. Finally, the desensitized YUV image is video-encoded to obtain an H264 or H265 file for disk storage. Data acquisition equipment trains its own model, and the model's reasoning ability needs to be comparable to that of the perception system chip. Otherwise, the desensitization effect will be significantly different from that of ADAS equipment. This requires high-end configuration of data acquisition equipment, which disrupts the balance between data acquisition performance, data desensitization capability, and cost.

[0029] Based on this, researchers discovered that the new approach, when applied to image desensitization in driver assistance systems, can effectively solve existing technical problems.

[0030] To enable those skilled in the art to better understand the technical concept, implementation scheme and beneficial effects of the embodiments of this application, detailed descriptions are provided below through specific embodiments.

[0031] Example 1

[0032] One embodiment of this application provides an assisted driving system that supports image desensitization, which can both perform image desensitization processing and reduce the cost of data acquisition equipment.

[0033] Figure 1 This is a schematic diagram of an assisted driving system supporting image desensitization in an embodiment of the present invention, as shown below. Figure 1As shown, the assisted driving system of this embodiment includes: a camera module 1, used to acquire images of a target area to obtain first original image data of the target area; an image data copying unit 2, connected to the camera module 1, used to copy the first original image data to obtain first copied image data and second copied image data; an assisted driving unit 3, connected to the image data copying unit 2, used to receive the first copied image data, perform first image processing on the first copied image data to obtain first YUV image data, and perform desensitization region detection on the first YUV image data to obtain desensitization region information; and a data acquisition unit 4, connected to the image data copying unit 2 and the assisted driving unit 3. The system is connected separately and is used to receive the second copied image data from the image data copying unit 2, perform second image processing on the second copied image data to obtain second YUV image data, and receive the desensitized area information from the assisted driving unit 3, and perform desensitization processing on the second YUV image data using the desensitized area information; wherein, the aspect ratio of the second YUV image represented by the second YUV image data is the same as the aspect ratio of the first YUV image represented by the first YUV image data, and the length and width dimensions of the second YUV image represented by the second YUV image data are the same as or at a predetermined ratio to the length and width dimensions of the first YUV image represented by the first YUV image data.

[0034] Camera modules are crucial electronic components for image capture. They enable optical imaging, photography, information capture and analysis, and visual interaction. Automotive camera assemblies typically consist of optical lenses, image sensors, ISPs, serializers, and connectors. With the continuous iteration of ADAS (Advanced Driver Assistance Systems) technologies, camera modules have evolved from 1V and 6V to 10V, with an increasing number of modules and continuously improving resolution. For example, module resolution has increased from 120W to 200W, and then to 800W, resulting in increasingly higher image data clarity and placing ever-higher demands on the technical capabilities of data acquisition equipment.

[0035] In this embodiment of the invention, the camera module is powered on and the device is initialized. After receiving the exposure request, the optical lens captures an image in the target area and projects the image onto the image sensor. After processing, the first raw image data of the target area is obtained. The target area can be the area of ​​the scene where the vehicle is located in real life. The first raw image data can also be called a RAW image.

[0036] The image data copying unit is connected to the camera module and copies the first original image data to obtain the first copied image data and the second copied image data. The first copied image data and the second copied image data are then transmitted to the driver assistance unit and the data acquisition unit, respectively.

[0037] When the driver assistance unit (ADAS) begins its normal operation, it sends an exposure request to the camera module. The camera module serializes the first raw image data and transmits it to the image data copying unit via a communication protocol. In some examples, the communication protocol is GMSL (Gigabit Multimedia Serial Links). The image copying unit copies the serialized first raw image data, resulting in two serialized data sets: a first copied image data set and a second copied image data set. The ADAS and the image data copying unit are connected, and the ADAS receives the first copied image data from the image data copying unit. The first copied image data undergoes first image processing to obtain first YUV image data. This first YUV image data is then input into an internal desensitization model for sensitive area detection to obtain desensitized area information.

[0038] The data acquisition unit is connected to the image data copying unit and the driver assistance unit, respectively. It receives second copied image data from the image data copying unit, performs second image processing on the second copied image data, and obtains second YUV image data. Since the driver assistance unit receives desensitized area information, it performs desensitization processing on the second YUV image data accordingly. Because the first and second copied image data originate from the same image, but the image signal processor (ISP) used for the first and second image processing is not the same device, the aspect ratios of the first and second YUV image data are the same, although some image details may differ, such as edge details and color. In other words, if the aspect ratio of the second YUV image represented by the second YUV image data is the same as or proportional to the aspect ratio of the first YUV image represented by the first YUV image data, then the desensitized area information inferred from the first YUV image data in the driver assistance unit can be used for the second YUV image data.

[0039] The embodiments of the present invention provide an assisted driving system, assisted driving control method, and vehicle supporting image desensitization. A data acquisition unit is connected to an image data copying unit and an assisted driving unit, respectively. The data acquisition unit receives second copied image data from the image data copying unit, performs second image processing on the second copied image data to obtain second YUV image data, and receives desensitized region information from the assisted driving unit, using the desensitized region information to perform desensitization processing on the second YUV image data. In this way, the data acquisition unit receives desensitized region information from the assisted driving unit and uses the desensitized region information to perform desensitization processing on the acquired image data. This eliminates the need for the data acquisition unit to run its own desensitization model, reducing the processor configuration required by the data acquisition unit and correspondingly lowering the cost of the data acquisition equipment.

[0040] In some embodiments, the data acquisition unit may use an FPGA (Field-Programmable Gate Array) chip combined with a microcontroller to interact with the driver assistance unit and achieve image desensitization.

[0041] In some embodiments, Figure 2 This is a schematic diagram of the driver assistance unit of a driver assistance system, such as... Figure 2 As shown, the assisted driving unit 3 includes: a first connection module 31 connected to the image data copying unit 2, used to receive the first copied image data; a first image processing module 32, used to perform first image processing on the first copied image data to obtain first YUV image data; and a desensitized region detection module 33, used to perform desensitized region detection on the first YUV image data through a desensitization model to obtain desensitized region information; or, used to detect information-sensitive regions in the first YUV image represented by the first YUV image data, assign a specified value to the Y value of the information-sensitive region to obtain an intermediate YUV image, subtract the Y value of the intermediate YUV image from the first YUV image to obtain a Y value difference map of the first YUV image, and take the non-zero region in the Y value difference map as the desensitized region.

[0042] The assisted driving unit 3 includes a first connection module 31, a first image processing module 32, and a desensitized area detection module 33. The three are interconnected and work together to obtain the desensitized information of the first original image data.

[0043] The first connection module 31 is connected at one end to the image copying unit 2 and at the other end to the first image processing module 32. It receives the first copied image data from the image copying unit 2 and transmits it to the first image processing module 32. After the assisted driving unit 3 is powered on, it performs device initialization. The first image processing module 32 performs first image processing on the first copied image data to obtain first YUV image data. In some examples, the first image processing module 32 includes an ISP. The first copied image data is a RAW image. After being converted into image signals in RGB, YUV, or other formats by the image signal processor ISP, the RAW image that cannot be directly analyzed is converted into an image that can be analyzed.

[0044] The first YUV image data is input to the desensitized region detection module 33, and the desensitization model is used to detect desensitized regions in the first YUV image data. After the desensitization model detects sensitive region information (such as faces or license plates) in the first YUV image data, it serializes and packages the sensitive region detection results (the position and size of faces and license plates in the first YUV image data) into a lightweight data exchange format (such as JSON format) or other serializable network order data BagDATA. For example, other serializable network order data BagDATA can be {"frameid":1,"type":"face","x":137,"y":855,"w":12,"h":57}.

[0045] The sensitive area detection result can be obtained directly by the driver assistance unit 3 through the desensitization model, or it can be obtained by the driver assistance unit 3 by calculating the difference between the pixel values ​​of the first YUV image data before and after desensitization. For example: assuming the first YUV image data is Y1, and the first YUV image data after desensitization is Y1c, where Y1 is a 4*4 pixel YUV image, and the YUV values ​​are as follows... Figure 3 As shown, the sensitive area is in the center of the image, and the YUV values ​​are blurred to 127 values, resulting in Y1c as shown. Figure 4 As shown, the difference between the YUV values ​​of the two images is obtained. Figure 5 The difference graph shows that the non-zero values ​​represent the desensitization results.

[0046] In some embodiments, Figure 6 This is a schematic diagram of the data acquisition unit of an assisted driving system, such as... Figure 6 As shown, the data acquisition unit 4 includes: a second connection module 41 connected to the image data copying unit 2 for receiving the second copied image data; a third connection module 42 connected to the assisted driving unit 3 for receiving the desensitized area information; a second image processing module 43 for performing second image processing on the second copied image data to obtain second YUV image data; and a desensitization processing module 44 for using the desensitized area information to perform desensitization processing on the second YUV image data.

[0047] The data acquisition unit 4 includes a second connection module 41, a third connection module 42, a second image processing module 43, and a desensitization processing module 44. The three are interconnected and work together to desensitize the first original image data.

[0048] The second connection module 41 is connected to the image copying unit 2 at one end and to the second image processing module 43 at the other end. It receives the second copied image data from the image copying unit 2 and transmits it to the second image processing module 43. The second image processing module 43 performs second image processing on the second copied image data to obtain second YUV image data. In some examples, the second image processing module 43 includes an ISP-like module, which is not the same as the ISP of the first image processing module 32. The second copied image data is a RAW image, which is converted into image signals in RGB, YUV, or other formats by the image signal processor-like ISP and then output.

[0049] The third connection module 42 is connected to the de-identification processing module 44 at one end and to the assisted driving unit 3 at the other end. After power-on, the communication middleware in the data acquisition unit 4 establishes a TCP (Transmission Control Protocol) connection with the satellite server of the assisted driving unit 3 as a client. It then maintains a fixed-time heartbeat to confirm the working status of both parties, enabling the data acquisition unit 4 and the assisted driving unit 3 to interact. The data acquisition unit 4 receives the de-identified area information from the assisted driving unit 3 through a communication protocol. In some examples, the communication protocol can be interactive, such as CAN (Controller Area Network) communication protocol bus, Ethernet, etc. After obtaining the de-identification result, the assisted driving unit 3 sends BagDATA to the de-identification processing module 44 in the data acquisition unit 4 through a custom TCP protocol.

[0050] The second YUV image data and the corresponding frame identifier information are transmitted to the video data desensitization processing module 44 of the data acquisition unit 4. The desensitization processing module 44 arranges the data information into the data queue to be desensitized. After receiving the desensitized area information BagDATA from the assisted driving unit 3, the data acquisition unit 4 parses the content to obtain the frame identifier information and uses the frame identifier information to retrieve the corresponding second YUV image data in the data queue. After obtaining the corresponding second YUV image data, since the data acquisition unit 4 uses an FPGA in some embodiments of the present invention, the data acquisition unit 4 can use its internal FPGA module to perform Gaussian or other blurring processing on the area corresponding to the data acquisition unit 4 that needs to be desensitized, and obtain the desensitized second YUV image data after processing.

[0051] In some embodiments, the data acquisition unit further includes: an image encoding module for encoding the desensitized second YUV image data to obtain an image encoding file in a predetermined format; and an encoding file saving module for saving the image encoding file to a specified location.

[0052] The data acquisition unit includes an image encoding module and an encoded file saving module. After performing desensitization operations on the picture data, it is also necessary to encode and save the picture data. In the image encoding module, the desensitized second YUV image data is transmitted to an internal video encoder for encoding. In the encoded file saving module, the encoded image file after encoding is saved to a specified location (in a database or other storage device). In some examples, the image encoding file can be an h264 or h265 bitstream file, which is a file format after processing by an efficient intra-frame and inter-frame compression algorithm.

[0053] In some embodiments, the data acquisition unit further includes: an encoded file acquisition module for acquiring the image encoding file from the specified location; an image decoding module for decoding the image encoding file to obtain an image decoding file in a predetermined format; and a third image processing module for performing third image processing on the image decoding file to obtain class raw image data corresponding to the image decoding file, so as to send the class raw image data to the assisted driving unit.

[0054] After saving the image encoding file, a decoding operation should also be performed. After power-on, the data acquisition unit operates in the backfill mode, and the client in the data acquisition unit does not actively connect to the server of the assisted driving unit.

[0055] The data acquisition unit includes an encoded file acquisition module, an image decoding module, and a third image processing module. In the encoded file acquisition module, the image encoding file (h264 or h265 bitstream file) is loaded from a specified location (in a database or other storage device). In the image decoding module, the image encoding file is decoded by a decoder into the already desensitized second YUV image data to obtain an image decoding file in a predetermined format. In the third image processing module, the image decoding file is input into an image signal processor supporting the second image processing module of the data acquisition unit for third image processing, and the image decoding file is processed into class raw image data in the same format as the camera module. After serializing the processed class raw image data, it is transmitted to the assisted driving unit through the communication protocol GMSL.

[0056] The third image processing and the foregoing second image processing are inverse processes. The second image processing ISP is to obtain the second YUV image data by using the red, blue, and green values of the photosensitive sensor and combining with a sampling algorithm to obtain the specific YUV value of each point from the first raw image data (RAW image) in a predetermined format (such as Bayer format) obtained by the exposure of the photosensitive sensor of the camera module. Therefore, knowing this sampling algorithm in the third image processing, it is possible to use each pixel point of the second YUV image data to inversely calculate the corresponding Bayer format value of the pixel point to obtain the class raw image data RAW image.

[0057] In some embodiments, the third image processing module is further configured to perform third image processing on the class of original image data to obtain third YUV image data corresponding to the class of original image data; wherein, the assisted driving unit further includes: a perception module configured to perform perception based on the third YUV image data to obtain perception results.

[0058] The driver assistance unit also includes a perception module. After the raw image data is transmitted to the driver assistance unit, the raw image data is processed by the third image processing module to obtain the third YUV image data corresponding to the raw image data. The third YUV image data is then transmitted to the perception module to obtain the perception result.

[0059] The perception model in the perception module is equivalent to running a large-scale image recognition algorithm. In some cases, the perception model can be based on two strategies: rule-based and AI-based. The rule-based strategy requires grayscale processing of the third YUV image data and convolutional matching of target attributes based on edge information. For example, if the third YUV image data is a license plate, the matching result is the license plate, and the output is the location of the license plate in the image. The AI-based strategy requires an image of an elephant as input for training, continuously iterating the model parameters, manually labeling the region location categories, inputting the third YUV image data, and outputting the corresponding location to obtain the result.

[0060] The driver assistance unit transmits the perception results to downstream operations (such as fusion algorithms and control algorithms) to achieve the purpose of data feedback from the data acquisition unit.

[0061] The desensitization results of the third YUV image data are highly consistent with those of the driver assistance unit. The desensitization action of the data acquisition unit uses the desensitization information provided by the desensitization model running within the driver assistance unit. Therefore, the desensitization results are very similar to the desensitization results obtained by the perception model at the time of acquisition, and have minimal impact on the perception results of the perception model. Thus, when this part of the data is fed back to the perception model of the driver assistance unit on the test bench, the inference results of the perception model are very similar to the perception results of the perception model of the driver assistance unit at the time of acquisition, which is conducive to reproducing perception problems.

[0062] Example 2

[0063] This application provides an embodiment of an assisted driving control method that supports image desensitization, which can perform image desensitization processing and effectively reduce the cost of data acquisition equipment. This assisted driving control method can be applied to the assisted driving control system that supports image desensitization described in the foregoing embodiments.

[0064] Figure 7 This is a flowchart illustrating an embodiment of the assisted driving control method supporting image desensitization provided in this application, as shown below. Figure 7 As shown, an assisted driving control method supporting image desensitization in this embodiment includes:

[0065] S11. Acquire images of the target area to obtain the first raw image data of the target area;

[0066] In this embodiment of the invention, the camera module is powered on and the device is initialized. After receiving an exposure request, the optical lens captures an image in the target area and projects the image onto the image sensor. After processing, the first raw image data of the target area is obtained. The target area can be the area of ​​the scene where the vehicle is located in real life, and the first raw image data can be a RAW image.

[0067] S12. Copy the first original image data to obtain the first copied image data and the second copied image data;

[0068] The first original image data is copied to obtain the first copied image data and the second copied image data, and the first copied image data and the second copied image data are respectively passed to the relevant modules.

[0069] S13, the assisted driving unit performs first image processing on the first copied image data to obtain first YUV image data, and performs desensitization region detection on the first YUV image data to obtain desensitization region information;

[0070] When the driver assistance unit (ADAS) begins its normal operation, it sends an exposure request to the camera module. The camera module serializes the first raw image data and transmits it to the image data copying unit via a communication protocol (GMSL in some examples). The image copying unit copies the serialized first raw image data, resulting in two serialized data sets: a first copied image data set and a second copied image data set. The ADAS and the image data copying unit are connected, and the ADAS receives the first copied image data from the image data copying unit. The first copied image data undergoes first image processing to obtain first YUV image data. This first YUV image data is then input into an internal desensitization model for sensitive area detection to obtain desensitized area information.

[0071] S14. The data acquisition unit performs second image processing on the second copied image data to obtain second YUV image data, and receives the desensitized area information from the assisted driving unit, and uses the desensitized area information to perform desensitization processing on the second YUV image data; wherein, the aspect ratio of the second YUV image represented by the second YUV image data is the same as the aspect ratio of the first YUV image represented by the first YUV image data, and the length and width dimensions of the second YUV image represented by the second YUV image data are the same as or at a predetermined ratio to the length and width dimensions of the first YUV image represented by the first YUV image data.

[0072] The data acquisition unit is connected to the image data copying unit and the driver assistance unit, respectively. It receives second copied image data from the image data copying unit and performs second image processing on the second copied image data to obtain second YUV image data. Since the driver assistance unit receives desensitized area information, it performs desensitization processing on the second YUV image data accordingly. Because the first and second copied image data originate from the same image, but the first and second image processing are performed using different devices, the aspect ratios of the first and second YUV image data are the same, although some image details may differ, such as edge details and color. In other words, if the aspect ratio of the second YUV image represented by the second YUV image data is the same as or proportional to the aspect ratio of the first YUV image represented by the first YUV image data, then the desensitized area information inferred from the first YUV image data in the driver assistance unit can be used for the second YUV image data.

[0073] An embodiment of the present invention provides an assisted driving control method supporting image desensitization, comprising: a data acquisition unit, performing second image processing on second copied image data to obtain second YUV image data, and receiving desensitized region information from an assisted driving unit, and using the desensitized region information to perform desensitization processing on the second YUV image data. In this way, the data acquisition unit receives desensitized region information from the assisted driving unit and uses the desensitized region information to perform desensitization processing on the acquired image data, eliminating the need for the data acquisition unit to run a desensitization model itself, thus reducing the processor configuration of the data acquisition equipment and correspondingly lowering the cost of the data acquisition unit.

[0074] In some embodiments, the step of detecting desensitized regions in the first YUV image data to obtain desensitized region information includes: detecting desensitized regions in the first YUV image data using a desensitization model to obtain desensitized region information; or detecting information-sensitive regions in the first YUV image represented by the first YUV image data, assigning a specified value to the Y value of the information-sensitive region to obtain an intermediate YUV image, subtracting the Y value of the intermediate YUV image from that of the first YUV image to obtain a Y value difference map of the first YUV image, and using the non-zero regions in the Y value difference map as desensitized regions.

[0075] The first YUV image data is input into the desensitized region detection module, and the desensitization model performs desensitization region detection on the first YUV image data. After the desensitization model detects sensitive region information (such as faces or license plates) in the first YUV image data, it serializes and packages the sensitive region detection results (the position and size of faces and license plates in the first YUV image data) into a lightweight data exchange format (such as JSON format) or other serializable network order data BagDATA. For example, other serializable network order data BagDATA can be {"frameid":1,"type":"face","x":137,"y":855,"w":12,"h":57}.

[0076] The sensitive area detection result can be obtained directly by the driver assistance unit through the desensitization model, or it can be obtained by the driver assistance unit by calculating the difference between the pixel values ​​of the first YUV image data before and after desensitization. For example: assuming the first YUV image data is Y1, and the first YUV image data after desensitization is Y1c, where Y1 is a 4*4 pixel YUV image, and the Y value is as follows... Figure 3 As shown, the sensitive area is in the center of the image, and the Y value is blurred to 127, resulting in Y1c as shown. Figure 4 As shown, the difference between the Y values ​​of the two images is obtained. Figure 5 The difference graph shows that the non-zero values ​​represent the desensitization results.

[0077] In some embodiments, the data acquisition unit receives the desensitized region information from the driver assistance unit, including: the data acquisition unit receives the desensitized region information from the driver assistance unit based on a communication protocol, and performs desensitization processing on the second YUV image data according to the desensitized region information.

[0078] The data acquisition unit and the driver assistance unit are connected via wires. After power-on, the communication middleware in the data acquisition unit establishes a TCP (Transmission Control Protocol) connection with the same satellite server of the driver assistance unit as a client. It then maintains a heartbeat at fixed intervals to confirm the working status of both parties, enabling interaction between the data acquisition unit and the driver assistance unit. The data acquisition unit receives the anonymized area information from the driver assistance unit through a communication protocol. In some examples, the communication protocol can be interactive, such as CAN (Controller Area Network) bus, Ethernet, etc. After obtaining the anonymized result, the driver assistance unit sends BagDATA to the data acquisition unit via a custom TCP protocol.

[0079] During the data acquisition process, the data acquisition unit directly obtains the desensitized information given by the perception results in the assisted driving unit through the interaction protocol. This desensitized information is used to perform blurring and desensitization processing on the video data. In some cases, the blurring and desensitization processing can be performed using an encoder, which is efficient and low-occupancy, and can reduce the processor configuration of the data acquisition equipment.

[0080] In some embodiments, after the data acquisition unit desensitizes the second YUV image data using the desensitized region information, the method further includes: encoding the desensitized second YUV image data to obtain an image encoding file of a predetermined format; saving the image encoding file to a specified location; retrieving the image encoding file from the specified location; decoding the image encoding file to obtain an image decoding file of a predetermined format; performing third image processing on the image decoding file to obtain near-raw image data corresponding to the image decoding file; sending the near-raw image data to the assisted driving unit; the assisted driving unit performing third image processing on the near-raw image data to obtain third YUV image data corresponding to the near-raw image data; and performing perception based on the third YUV image data to obtain a perception result.

[0081] After desensitizing the second YUV image data, the image data still needs to be encoded and saved. The desensitized second YUV image data is transmitted to an internal video encoder for encoding. The encoded image file is then saved to a specified location (database or other storage device). In some cases, the image file can be an h264 or h265 bitstream file, which is a file format processed by efficient intra-frame and inter-frame compression algorithms.

[0082] After saving the image encoding file, a decoding operation should also be performed. After powering on, the data acquisition unit should be put into recharge mode, and the client in the data acquisition unit and the server of the assisted driving unit should not actively connect.

[0083] The image encoding file (h264 or h265 bitstream file) is loaded from a specified location (database or other storage device). The image encoding file is decoded by a decoder into desensitized second YUV image data, resulting in an image decoding file of a predetermined format. The image decoding file is then input to an image signal processor for third-party image processing. This process converts the image decoding file into near-raw image data consistent with the format of the camera module. The processed near-raw image data is then serialized and transmitted to the driver assistance unit via the GMSL communication protocol.

[0084] The third image processing step is the inverse of the second image processing step. The second image processing (ISP) takes the first raw image data (RAW image) obtained by the camera module's photosensitive sensor in a predetermined format (such as Bayer format), and uses the red, blue, and green values ​​of the photosensitive sensor, along with a sampling algorithm, to obtain the specific YUV values ​​for each point, thus obtaining the second YUV image data. Therefore, the third image processing step, knowing this sampling algorithm, can use each pixel of the second YUV image data to calculate the corresponding Bayer format value for that pixel, obtaining a RAW image similar to the raw image data.

[0085] After the original image data is transmitted to the driver assistance unit, the first image processing module performs third image processing on the original image data to obtain third YUV image data corresponding to the original image data. The third YUV image data is then used for perception to obtain the perception result.

[0086] The perceptual model used for sensing third YUV image data is equivalent to running a large-scale image recognition algorithm. In some cases, the perceptual model can be based on two strategies: rule-based and AI-based. Rule-based strategies require grayscale processing of the third YUV image data and convolutional matching of target attributes based on edge information. For example, if the third YUV image data represents a license plate, the matching result is the license plate, and the output is the location of the license plate in the image. AI-based large-scale models require input images of elephants for training, iterating model parameters continuously, manually labeling region location categories, inputting third YUV image data, and outputting the corresponding location to obtain the result.

[0087] To illustrate the above operation process, the data acquisition unit and the assisted driving unit equipped with this system encountered a passenger car V1 with a license plate and a pedestrian P1 traveling in the opposite direction during data acquisition. During operation, the data acquisition unit acquired the second YUV image data Y2 at this moment, and the assisted driving unit acquired the first YUV image data Y1 at this moment, obtaining the anonymized information in Y1. In the anonymized information, the coordinates of P1's face in the image are 1168, 1287, 1195, 1271 (top, right, bottom, left, the same parameters below), and the coordinates of V1's license plate in the image are 1213, 2203, 1226, 2174. The driver assistance unit transmits these two sets of desensitized information to the data acquisition unit via TCP. The data acquisition unit performs Gaussian or other blurring operations on the YUV values ​​of the corresponding sensitive positions in the image Y2 at this moment to obtain the desensitized Y2c. Then, Y2c is transmitted to the internal video encoder for encoding. The encoded image file is saved to the specified location (database or other storage device). The video data stored at this time is the desensitized data.

[0088] When it is necessary to reload the scene for problem reproduction or performance analysis, the data acquisition unit switches to reload mode, loads the image encoding file (h264 or h265 bitstream file) from the specified location (database or other storage device), decodes the image encoding file into desensitized second YUV image data through the decoder, and obtains the image decoding file in the predetermined format. After decoding, Y2c is obtained. At this time, because the sensitive areas in the image have been Gaussianized or zeroed out, the information is unrecoverable. At the same time, because the desensitized area of ​​the image is the algorithm provided by the assisted driving unit itself, the negative impact of the desensitized image when it is used for inference by the perception model can be minimized.

[0089] Example 3

[0090] Embodiments of the present invention also provide a vehicle equipped with an assisted driving system, wherein the assisted driving system is any of the assisted driving systems provided in the foregoing embodiments.

[0091] An embodiment of the present invention provides a vehicle equipped with an assisted driving system. The assisted driving system includes: a data acquisition unit connected to an image data copying unit and the assisted driving unit, respectively; a data acquisition unit receiving second copied image data from the image data copying unit; performing second image processing on the second copied image data to obtain second YUV image data; and receiving desensitized region information from the assisted driving unit, and using the desensitized region information to perform desensitization processing on the second YUV image data. In this way, the data acquisition unit receives desensitized region information from the assisted driving unit and uses the desensitized region information to perform desensitization processing on the acquired image data. This eliminates the need for the data acquisition unit to run its own desensitization model, reducing the processor configuration required by the data acquisition unit and correspondingly lowering the cost of the data acquisition equipment.

[0092] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0093] The various embodiments in this specification are described in a related manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0094] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An assisted driving system supporting image desensitization, characterized by, The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device.

2. The system of claim 1, wherein, The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device.

3. The system of claim 1, wherein, The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device.

4. The system of claim 3, wherein, The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device.

5. The system of claim 4, wherein, The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing method and device. The application relates to an image data processing The method comprises the following steps: an image decoding module is configured to decode the image coding file to obtain an image decoding file in a predetermined format; a third image processing module is configured to perform third image processing on the image decoding file to obtain raw image-like data corresponding to the image decoding file, and send the raw image-like data to the auxiliary driving unit.

6. The system of claim 5, wherein, The third image processing module is further configured to perform third image processing on the raw image-like data to obtain third YUV image data corresponding to the raw image-like data. The auxiliary driving unit further comprises: a perception module is configured to perform perception based on the third YUV image data to obtain a perception result.

7. An assisted driving control method supporting image desensitization, characterized by, The method comprises the following steps: perform image acquisition on a target region to obtain first raw image data of the target region; copy the first raw image data to obtain first copy image data and second copy image data; an auxiliary driving unit is configured to perform first image processing on the first copy image data to obtain first YUV image data, and perform desensitization region detection on the first YUV image data to obtain desensitization region information; a data acquisition unit is configured to perform second image processing on the second copy image data to obtain second YUV image data, and receive the desensitization region information from the auxiliary driving unit, and perform desensitization processing on the second YUV image data by using the desensitization region information; wherein a length-width ratio of a second YUV image represented by the second YUV image data is the same as a length-width ratio of a first YUV image represented by the first YUV image data, and a length-width size of the second YUV image represented by the second YUV image data is the same as or in a predetermined proportion to a length-width size of the first YUV image represented by the first YUV image data.

8. The method of claim 7, wherein, The method further comprises the following steps: perform desensitization region detection on the first YUV image data by using a desensitization model to obtain desensitization region information; or detect an information sensitive region in the first YUV image represented by the first YUV image data, assign a specified value to Y values of the information sensitive region to obtain an intermediate YUV image, and subtract Y values of the first YUV image from the intermediate YUV image to obtain a Y value difference map of the first YUV image, and regard a region other than 0 in the Y value difference map as a desensitization region.

9. The method of claim 7, wherein, The method further comprises the following steps: The data acquisition unit receives the desensitization region information from the auxiliary driving unit based on a communication protocol, and performs desensitization processing on the second YUV image data according to the desensitization region information.

10. The method of claim 7, wherein, The method further comprises the following steps: perform encoding on the second YUV image data after the desensitization processing to obtain an image coding file in a predetermined format; save the image coding file to a specified location. acquire the image coding file from the specified location; decode the image coding file to obtain an image decoding file in a predetermined format, perform third image processing on the image decoding file to obtain raw image-like data corresponding to the image decoding file, and send the raw image-like data to the auxiliary driving unit; the auxiliary driving unit performs third image processing on the raw image-like data to obtain third YUV image data corresponding to the raw image-like data, and performs perception based on the third YUV image data to obtain a perception result.

11. A vehicle on which an assist system is provided, characterized by The auxiliary driving system is the auxiliary driving system according to any one of the preceding claims 1-6.

Images