Image data processing method and device for automatic parking scene, equipment and medium

Abstract

The invention discloses an image data processing method and device for an automatic parking scene, equipment and a medium. Comprising the following steps: acquiring current frame image data which is not a first frame image data; performing interpolation processing on the current frame image data and the previous frame image data based on a preset frame interpolation algorithm to obtain frame interpolation image data between the current frame image data and the previous frame image data; under the condition that the current frame of image data is the second frame of image data, image drawing is conducted on the frame insertion image data based on preset delay time, and the preset delay time is the generation time of each frame of image data of the parked vehicle; and under the condition that the current frame image data is not the second frame image data, performing image drawing on the frame insertion image data and the current frame image data according to a fixed time interval. According to the method and the device, the serious problems of unsmooth parking picture, jamming and the like can be solved, so that the automatic parking rendering picture is smoother and more real, and the user experience is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image data processing technology, and in particular to a method, device, equipment and medium for processing image data of an automatic parking scene. Background Art

[0002] Automatic parking refers to the automatic parking system of cars, which can automatically park the vehicle into the parking space without human intervention. Through the rendering of the automatic parking screen, users can intuitively observe the movement trajectory of the vehicle, the location and size of the parking space, and the distance from obstacles, thereby increasing their trust and understanding of the automatic parking system.

[0003] At present, APA (Automatic Parking Assist) is usually used to assist vehicles in automatic parking. APA is an advanced driver assistance technology that uses sensors around the vehicle (such as radar, cameras, etc.) to detect parking spaces and surrounding obstacles, and helps drivers to automatically park in and out of the vehicle through complex algorithms and control systems. The data input for APA rendering relies on the parking ECU (Electronic Control Unit) and the IP communication protocol SomeIP (Scalable service-Oriented MiddlewarE over IP), and the data input frequency can only reach 10 frames per second. When receiving this low-frequency data for APA screen rendering, it will cause serious problems such as unsmooth screen and freezes, which greatly affects the user experience. Summary of the invention

[0004] The embodiments of the present application provide an image data processing method, device, equipment and medium for an automatic parking scene, which can be used to solve serious problems such as unsmooth and stuck parking images, making the automatic parking rendering images smoother and more realistic, and improving the user experience. The technical solution is as follows:

[0005] On the one hand, an embodiment of the present application provides an image data processing method for an automatic parking scene, the method comprising:

[0006] Acquire current frame image data, where the current frame image data is not the first frame image data;

[0007] Performing interpolation processing on the current frame image data and the previous frame image data corresponding to the current frame image data based on a preset interpolation algorithm to obtain interpolation frame image data between the current frame image data and the previous frame image data;

[0008] In a case where the current frame image data is the second frame image data, image rendering is performed on the interpolated frame image data based on a preset delay time, wherein the preset delay time is a generation time of each frame image data of the parked vehicle;

[0009] In a case where the current frame image data is not the second frame image data, image rendering is performed on the interpolated frame image data and the current frame image data at a fixed time interval.

[0010] On the other hand, an embodiment of the present application provides an image data processing device for an automatic parking scene, the device comprising:

[0011] A current frame data acquisition module, used to acquire current frame image data, where the current frame image data is not the first frame image data;

[0012] An interpolation frame data acquisition module, configured to perform interpolation processing on the current frame image data and the previous frame image data corresponding to the current frame image data based on a preset interpolation frame algorithm to obtain interpolation frame image data between the current frame image data and the previous frame image data;

[0013] an interpolation frame data processing module, configured to, when the current frame image data is the second frame image data, perform image drawing on the interpolation frame image data based on a preset delay time, wherein the preset delay time is a generation time of each frame image data of the parked vehicle;

[0014] The image data processing module is used to draw images on the interpolated frame image data and the current frame image data at a fixed time interval when the current frame image data is not the second frame image data.

[0015] On the other hand, an embodiment of the present application provides a computer device, which includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the method described in the above aspects.

[0016] On the other hand, an embodiment of the present application provides a computer-readable storage medium, in which at least one instruction, at least one program, a code set or an instruction set is stored, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the method described in the above aspects.

[0017] On the other hand, an embodiment of the present application provides a computer program product, which, when executed, is used to execute the method described in the above aspects.

[0018] In the technical solution provided by the embodiment of the present application, by acquiring the current frame image data, the current frame image data is not the first frame image data. Based on the preset interpolation algorithm, the current frame image data and the previous frame image data corresponding to the current frame image data are interpolated to obtain the interpolated frame image data between the current frame image data and the previous frame image data. When the current frame image data is the second frame image data, the interpolated frame image data is drawn based on the preset delay time, and the preset delay time is the generation time of each frame image data of the parked vehicle. When the current frame image data is not the second frame image data, the interpolated frame image data and the current frame image data are drawn at a fixed time interval. In the embodiment of the present application, by using the preset interpolation algorithm to interpolate between the current frame and the previous frame image data in the automatic parking scenario, the parking picture generated per second during automatic parking can be improved, and serious problems such as unsmooth parking pictures and freezes can be solved, so that the automatic parking rendering is smoother and more realistic, and the user experience is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A flowchart of a method for processing image data of an automatic parking scene provided in an embodiment of the present application;

[0020] Figure 2 An architectural diagram of a parking image data processing product provided in an embodiment of the present application;

[0021] Figure 3 A schematic diagram of an image data processing flow provided in an embodiment of the present application;

[0022] Figure 4 A schematic diagram of the structure of an image data processing device for an automatic parking scene provided in an embodiment of the present application;

[0023] Figure 5 A structural block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0025] Reference Figure 1 , shows a flowchart of the steps of an image data processing method for an automatic parking scene provided by an embodiment of the present application. Figure 1 As shown, the image data processing method of the automatic parking scene may include: step 101, step 102, step 103 and step 104.

[0026] Step 101: Acquire current frame image data, where the current frame image data is not the first frame image data.

[0027] The embodiments of the present application may be applied to a scenario in which a preset number of parking images are drawn per second during an automatic parking process.

[0028] First, you can combine Figure 2 The hardware content of the product is described as follows.

[0029] like Figure 2 As shown, the product hardware may include: parking ECU, Java components and Native components. Among them, Java components may include: SomeIp Middleware and APP, among them, APP may include: SomeIp Analysis Client, Javasdk, Javasdk includes: Java APA / VPA Render SDK, Java UISDK, Videocat.jar aidlClient. Native components may include: SomeIp Server, Videocat.jar carcam server, CarcamUISDK and Kanzi Engine Render.

[0030] Among them, the parking ECU is the core controller of the parking system, which is responsible for receiving parking image data and other related information from sensors (such as cameras, radars, etc.), processing and analyzing the data, and then controlling the actuators (such as motors, steering gears, etc.) to complete the parking operation.

[0031] SomeIp Middleware (Service-Oriented Extensible IP Middleware) is SomeIp middleware. When processing parking image data, SomeIp middleware is responsible for providing network communication functions, enabling Java applications to transmit and interact with Native components (such as SomeIp Server). It supports communication mechanisms such as service discovery and request response to ensure that data is correctly transmitted in the system.

[0032] The APP (application) is the user interface and control center of the parking system, which may contain multiple subcomponents to support the processing and analysis of parking image data.

[0033] SomeIp Analysis Client is used to monitor and analyze image data during parking in real time, and provide functions such as data visualization and anomaly detection.

[0034] Java SDK (Java Software Development Kit) provides a series of tools and libraries for parking image data processing.

[0035] Java APA / VPA Render SDK (APA / VPA Rendering Software Development Kit), which supports the image rendering functions required by the APA / VPA system, such as superimposing parking path, obstacles and other information on the real-time image to provide intuitive parking guidance.

[0036] The Java UISDK (User Interface Software Development Kit) provides tools and components for building parking system user interfaces, such as buttons, sliders, text boxes, etc., as well as interface elements for displaying parking images and results.

[0037] Videocat.jar aidl Client (Videocat.jar AIDL client) is used to communicate with Videocat.jar server to send parking image data. AIDL is an Android technology used to define cross-process communication interfaces and can be used for communication between Java components and Native components.

[0038] When SomeIp Server processes parking image data, SomeIp Server acts as a communication center of the Native component, receives image data from the parking ECU, and processes the image data.

[0039] Videocat.jar carcam server (Videocat.jar car camera server), this component is responsible for obtaining parking image data, performing preliminary processing (such as image enhancement, noise filtering, etc.), and then sending the processed data to the components that need it (such as SomeIp Server, Javasdk, etc.).

[0040] Carcam UISDK (Car Camera User Interface Software Development Kit) provides user interface elements for configuring and displaying car camera related information, such as camera selection, image preview, etc.

[0041] Kanzi Engine Render is a high-performance 3D rendering engine that may be used to provide high-quality 3D image rendering effects during the parking process, such as 3D parking path simulation, 3D obstacle display, etc. These rendering effects can significantly improve the user experience and safety of the parking system.

[0042] In this embodiment, the data rendered by APA originates from the parking ECU, which is a data stream generated by the parking ECU. These data are transmitted through SomeIP Sever (the party that provides services in the SomeIP communication protocol) and sent to SomeIP Middleware (a scalable service-oriented middleware running on IP) for further processing and distribution. When the APP (application) receives the perception data stream from SomeIP Middleware, it parses the data stream into the format of a Json string for easier processing and use.

[0043] In the above process, this embodiment is responsible for parsing and processing Json data to extract the required information and perform corresponding processing. At the same time, these data are used to generate intermediate frames to achieve a smooth animation transition effect. By generating intermediate frames between the current frame and the previous frame, the animation transition rendered by APA is ensured to be smoother.

[0044] Finally, the processed and generated APA rendering data will be sent down and displayed on the screen by the rendering engine in the Native component. In this way, users can get visual feedback related to parking by watching the rendering effect on the screen.

[0045] During the parking process, the current frame image data may be obtained. In this example, the current frame image data may be the first frame image data or may be non-first frame image data. Specifically, it may be determined according to actual conditions, and this embodiment does not limit this.

[0046] When the current frame image data obtained is the first frame image data (i.e., the first frame image data generated at the start of parking), the current frame image data can be written into a pre-established cache queue. At the same time, the current frame image data is drawn, that is, the current frame image data is rendered and displayed in real time.

[0047] When the current frame image data is not the first frame image data, step 102 is executed.

[0048] Step 102: performing interpolation processing on the current frame image data and the previous frame image data corresponding to the current frame image data based on a preset interpolation algorithm to obtain interpolation frame image data between the current frame image data and the previous frame image data.

[0049] The preset interpolation algorithm refers to an algorithm used to interpolate between two adjacent frames. The goal of the preset interpolation algorithm is to generate intermediate frames between known key frames (starting frame and ending frame) to make the animation transition smoother. Common interpolation algorithms include linear interpolation, Bezier curve interpolation, spline interpolation, etc. In this example, the preset interpolation algorithm is preferably a linear interpolation algorithm.

[0050] In the case where the current frame image data is not the first frame image data, the current frame image data and the previous frame image data corresponding to the current frame image data can be interpolated based on a preset interpolation algorithm to obtain interpolated image data between the current frame image data and the previous frame image data. Specifically, the interpolation example process can be: extracting the RGB (or YUV, etc.) value of each pixel from the current frame image data, and extracting the RGB (or YUV, etc.) value of the corresponding pixel from the previous frame image data. Then, determine the interpolation position. For example, if you want to generate an intermediate frame between the current frame and the previous frame, the interpolation coefficient can be 0.5 (indicating an intermediate position). The interpolation coefficient usually varies between 0 and 1, 0 means completely taking the value of the previous frame, 1 means completely taking the value of the current frame, 0.5 means taking the average of the two, and so on.

[0051] In this embodiment, the number of required images per second can be predefined, and the number of image frames to be interpolated between two adjacent frames of image data can be determined in combination with the image frame rate of the vehicle, and interpolation processing can be performed according to the number. The implementation process can be described in detail in combination with the following specific implementation methods.

[0052] In a specific implementation of the present application, the above step 102 may include:

[0053] Sub-step A1: Determine the number of image frames to be interpolated between two adjacent frames of image data according to a preset number of required images per second and the image frame rate of the vehicle.

[0054] In this embodiment, a preset required number of images per second and an image frame rate of the vehicle may be obtained.

[0055] The number of images required per second refers to the number of parking scene images displayed on the vehicle screen per second.

[0056] Image frame rate refers to the update frequency of the image sequence. For example, when the original data input is 100ms per frame, the output image is about 10 frames per second, then the image frame rate is 10 frames / second, etc.

[0057] When performing interpolation, the number of image frames to be interpolated between two adjacent frames of image data can be determined based on the preset number of required images per second and the image frame rate of the vehicle. For example, the original data input is 100ms per frame, and the output video is about ten frames. Since the data is not smooth enough, if the data is interpolated, in order to ensure that the target of 30 frames per second is achieved, two frames need to be inserted between two adjacent frames.

[0058] It can be understood that the above examples are merely examples listed for a better understanding of the technical solutions of the embodiments of the present application, and are not intended to be the sole limitation to the embodiments.

[0059] After determining the number of image frames to be interpolated between two adjacent frames of image data according to the preset required number of images per second and the image frame rate of the vehicle, sub-step A2 is performed.

[0060] Sub-step A2: using a linear interpolation algorithm to perform interpolation processing on the current frame image data and the previous frame image data to obtain the interpolated frame image data of the number of image frames.

[0061] After determining the number of image frames to be interpolated between two adjacent frames of image data based on the preset number of required images per second and the image frame rate of the vehicle, a linear interpolation algorithm can be used to interpolate the current frame image data and the previous frame image data to obtain interpolated frame image data of the same number of image frames.

[0062] The embodiment of the present application uses a linear interpolation algorithm to interpolate the current frame image data and the previous frame image data to obtain the required number of interpolated frame image data, and can generate additional image frames between two adjacent frames of images, thereby effectively improving the frame rate of the overall parking image, making the parking animation transition smoother, and improving the fluency and viewing experience of the video.

[0063] In this embodiment, when performing interpolation, parameters required for image drawing in the current frame image data and the previous frame image data can be obtained, and interpolation is performed according to the coordinates corresponding to the parameters. The implementation process can be described in detail in conjunction with the following specific implementation methods.

[0064] In another specific implementation of the present application, the above sub-step A2 may include:

[0065] Sub-step B1: according to the parking requirement information, obtaining a first parameter required for image rendering in the previous frame image data and a second parameter matching the first parameter in the current frame image data.

[0066] In this embodiment, the first parameter required for image drawing in the previous frame image data and the second parameter matching the first parameter in the current frame image data can be obtained according to the parking demand information. In this example, before the current frame image is interpolated with the previous frame image, it is necessary to ensure that the current frame image has a target with the same ID, which means that a target has changed position and / or orientation between the two frames. The first parameter and the second parameter are the parameters of the target in position and / or orientation.

[0067] When the first parameter and the second parameter are parameters of a parking space, the first parameter may be four parking space corner points of the parking space in the previous frame image data, and the second parameter may be four parking space corner points of the parking space in the current frame image data. For example, for a parking space, the x and y coordinates of the four corners may be taken, for a total of four parameters.

[0068] When the first parameter and the second parameter are parameters of an obstacle, the first parameter may be the center point of the obstacle in the previous frame of image data and the arc parameter of the obstacle relative to the vehicle, and the second parameter may be the center point of the obstacle in the current frame of image data and the arc parameter of the obstacle relative to the vehicle. For example, for an obstacle, three parameters may be taken, namely, x, y coordinates and an angle (i.e., the arc parameter in this example).

[0069] After obtaining the first parameter required for image drawing in the previous frame image data and the second parameter matching the first parameter in the current frame image data, sub-step B2 is performed.

[0070] Sub-step B2: using the linear interpolation algorithm to perform interpolation processing on the current frame image data and the previous frame image data according to the first parameter and the second parameter to obtain the interpolated frame image data.

[0071] After obtaining the first parameter needed for image drawing in the previous frame image data and the second parameter matching the first parameter in the current frame image data, a linear interpolation algorithm can be used to interpolate the current frame image data and the previous frame image data according to the first parameter and the second parameter to obtain interpolated frame image data. Specifically, all points that need to be drawn in the previous frame data can be traversed. If the point exists in both the previous frame data and the current frame data, the coordinate data is segmented and values ​​are taken at positions 1 / n, 2 / n, ..., (n-1) / n (where n is the number of interpolated frames), and the data of the inserted frame is updated. The inserted frame is stored in a queue. The values ​​of the first parameter and the second parameter can be used as reference standards for the coordinates or radian parameter values ​​of all points that need to be drawn, thereby achieving the values ​​of the targets that exist in two adjacent frames of image data, and then realizing the interpolation process.

[0072] Of course, in practical applications, for coordinate interpolation, linear interpolation, quadratic interpolation and other methods can be used to calculate the points in the intermediate frames. For arc and center point interpolation, rotation and translation interpolation can be considered to ensure that the obstacle shape and orientation after interpolation are correct.

[0073] The embodiment of the present application uses the values ​​of the first parameter and the second parameter as reference standards and combines them with appropriate interpolation methods to effectively achieve the values ​​of points that exist in two adjacent frames of image data and generate high-quality interpolated image data.

[0074] In the specific implementation, in order to avoid abnormal movement of objects, since the interpolation frame will take the value of the middle point, if the data changes abnormally, it may cause the object on the screen to float randomly. Therefore, the strategy that can be adopted is to update the data of the point to the data of the current frame if an abnormality is found. Specifically, it can include the following situations:

[0075] 1. Filtering of parking spaces: After the perception data is updated, if the parking space data is found to be jittery, filtering is performed. Specifically, when the current frame image data is not the first frame image data, and the first parameter and the second parameter are parameters of the parking space, the first interval distance between the first parameter and the corresponding second parameter can be obtained. When the first interval distance is greater than the first distance threshold, the current frame image data is not interpolated, and the current frame image data is used as the first frame image data for image drawing.

[0076] 2. Filtering of obstacles: Filter image frame data with large curvature change values ​​and / or long center point intervals. Specifically, when the current frame image data is not the first frame image data, and the first parameter and the second parameter are parameters of the obstacle, the second interval distance between the center point of the obstacle in the current frame image data and the center point of the obstacle in the previous frame image data, and the absolute value of the curvature change parameter between the curvature parameter corresponding to the current frame image data and the curvature parameter corresponding to the previous frame image data can be obtained. When the second interval distance is greater than the second interval distance threshold, and / or the absolute value of the curvature change parameter is greater than the curvature parameter threshold, the current frame image data is not interpolated, and the current frame image data is used as the first frame image data for image drawing.

[0077] The embodiment of the present application can avoid the phenomenon of objects floating randomly on the screen and improve the smoothness of the parking animation by filtering the image data with large data changes.

[0078] In this embodiment, a queue can be used to maintain the interpolation data. All the perception data (i.e., image data) and interpolation image data passed by the application will be stored in the queue. The implementation process can be described in detail in conjunction with the following specific implementation method.

[0079] In another specific implementation of the present application, the above step 102 may include:

[0080] Sub-step C1: reserving an image frame position for interpolated frame data in a pre-established cache queue, and writing the current frame image data into a position after the image frame position in the cache queue.

[0081] In this embodiment, when the current frame image data is acquired and the current frame image data is not the first frame image data, an image frame position for interpolated frame data can be reserved in a pre-established cache queue. The image frame position is the position for writing the interpolated frame image data between the current frame image data and the previous frame image data. At the same time, the current frame image data can be written to the position after the image frame position in the cache queue.

[0082] After reserving an image frame position for the interpolated frame data in the pre-established cache queue and writing the current frame image data into a position after the image frame position in the cache queue, sub-step C2 is performed.

[0083] Sub-step C2: performing interpolation processing on the current frame image data and the previous frame image data based on the preset interpolation algorithm to obtain the interpolation frame image data.

[0084] After reserving an image frame position for interpolation frame data in a pre-established cache queue and writing the current frame image data to a position after the image frame position in the cache queue, the current frame image data and the previous frame image data can be interpolated based on a preset interpolation algorithm to obtain interpolation frame image data.

[0085] After the current frame image data and the previous frame image data are interpolated based on a preset interpolation algorithm to obtain interpolation frame image data, sub-step C3 is executed.

[0086] Sub-step C3: writing the interpolated frame image data into the image frame position in the cache queue.

[0087] After the current frame image data and the previous frame image data are interpolated based on a preset interpolation algorithm to obtain the interpolated frame image data, the interpolated frame image data can be written into the image frame position in the cache queue.

[0088] The embodiment of the present application pre-sets a cache queue to cache received image frame data and interpolated image data and writes them into the cache queue in sequence according to the image display order, which can facilitate the subsequent reading of image data at fixed time intervals to achieve a smooth transition of the parking animation and improve the fluency and viewing experience of the parking animation.

[0089] After interpolating the current frame image data and the previous frame image data corresponding to the current frame image data based on a preset interpolation algorithm to obtain interpolated frame image data between the current frame image data and the previous frame image data, step 103 is executed, or step 104 is executed.

[0090] Step 103: when the current frame image data is the second frame image data, image rendering is performed on the interpolated frame image data based on a preset delay time, where the preset delay time is the generation time of each frame image data of the parked vehicle.

[0091] In the case where the current frame image data is the second frame image data, the interpolated frame image data can be drawn based on the preset delay time, and the preset delay time is the generation time of each frame image data of the parked vehicle. Specifically, the interpolated frame data processing time corresponding to the interpolated frame image data can be obtained, and the interpolated frame data processing time can be used as the preset delay time. Furthermore, the timing can be started from the moment when the current frame image data is received, and after the preset delay time, the pre-established thread is called to read the interpolated frame image data from the cache queue and draw the image.

[0092] In the embodiment of the present application, since the interpolation process requires additional calculation and processing time, there will be a certain delay in the drawing of the data. For example, in this process, the data delay is 100ms, that is, the interpolation data after the first frame of image data will be drawn after a delay of 100ms on the basis of the original data. Through such interpolation processing and delayed drawing, a smoother video effect can be obtained, which improves the user's viewing experience.

[0093] Step 104: When the current frame image data is not the second frame image data, image rendering is performed on the interpolated frame image data and the current frame image data at a fixed time interval.

[0094] In the case where the current frame image data is not the second frame image data, the interpolated frame image data and the current frame image data can be drawn at a fixed time interval. Specifically, the fixed time interval can be determined according to the preset number of required images per second. The pre-established thread is called to read the image data from the cache queue and draw the image. For example, the preset number of required images per second is 40 frames. At this time, one frame of image data can be read from the cache queue every 25 milliseconds for image drawing.

[0095] In this example, the pre-established thread may be a thread HandlerThread, which is specifically used to process interpolated frame data and perform drawing to ensure that a set number of image frames are drawn per second. The HandlerThread may be awakened by sending a message.

[0096] It can be understood that the above examples are merely examples listed for a better understanding of the technical solutions of the embodiments of the present application, and are not intended to be the sole limitation to the embodiments.

[0097] In actual applications, the frequency and stability of received data cannot be guaranteed due to system performance and network fluctuations. In this embodiment, the created perception data is sequentially stored at the tail of a double-ended queue, and the HandlerThread is continuously awakened at fixed time intervals to obtain the perception data from the queue, thereby achieving fixed-frequency rendering.

[0098] In this embodiment, when the amount of image data in the cache queue is large, a preset frame dropping strategy may be used to drop part of the image data in the cache queue. This implementation process may be described in detail in conjunction with the following specific implementation manner.

[0099] In a specific implementation of the present application, the method may further include:

[0100] Step D1: Obtain the number of image frames of the image data cached in the cache queue.

[0101] In this embodiment, the number of image frames of image data cached in the cache queue can be obtained. In a specific implementation, the number of image frames of image data in the cache queue can be obtained once every preset time, or the number of image frames of image data in the cache queue can be obtained once each time image frame data is written to the cache queue. Specifically, the time for obtaining the number of image frames of image data cached in the cache queue can be determined according to business needs, and this embodiment does not limit this.

[0102] After obtaining the number of image frames of the image data cached in the cache queue, step D2 is executed, or step D3 is executed.

[0103] Step D2: when the number of image frames is greater than a first multiple of the interpolated frame image data cached in the cache queue, discard the earliest frame of image data cached in the cache queue.

[0104] When the number of image frames of image data cached in the cache queue is greater than the number of image frames of image data cached in the cache queue, the earliest frame of image data cached in the cache queue may be discarded.

[0105] Step D3: When the number of image frames is greater than the second multiple of the interpolated image data cached in the cache queue, discard the N frames of image data cached earliest in the cache queue, where N is a positive integer and is half of the number of image frames.

[0106] When the number of image frames is greater than the second multiple of the interpolated frame image data cached in the cache queue, the N frames of image data cached earliest in the cache queue may be discarded, where N is a positive integer and is half of the number of image frames, and the second multiple is greater than the first multiple.

[0107] In actual applications, the frame drop strategy can be set in advance: due to the instability of writing speed and rendering speed, when the content of the data queue is too much, it will affect the drawing strategy. Therefore, some earlier data needs to be discarded to ensure the normal rendering of subsequent data. The frame drop strategy may include: 1. Threshold QUEUE_LEVEL_1: twice the number of interpolated frames; 2. Threshold QUEUE_LEVEL_2: four times the number of interpolated frames. When the number of data queues (i.e. cache queues) is greater than QUEUE_LEVEL_1, one frame of data is discarded; when the number of data queues is greater than QUEUE_LEVEL_2, half of the data is discarded.

[0108] The embodiment of the present application takes into account the performance factors of the vehicle computer and in order to ensure that the drawing can achieve a real-time low latency effect, each time the queue is read, data will be discarded from the head of the queue according to the amount of data stored in the queue.

[0109] In this embodiment, the perception data may not be continuous. In order to avoid the last long-ago data affecting the next interpolation, if no data is received within a preset time, the image data in the cache queue is cleared. Specifically, the timing can be started from the moment the current frame image data is received to determine whether the next frame image data is received within the preset timing time. If the next frame image data is not received, the image data cached in the cache queue can be cleared.

[0110] The embodiment of the present application resets the current interpolation state when no data is received, and re-executes the interpolation strategy according to the first frame processing for the next received data, thereby ensuring a high degree of match between the rendering and the actual scene.

[0111] Of course, the above solution provided in the embodiment of the present application can be applied not only to automatic parking scenarios, but also to scenarios such as game development and application simulation.

[0112] The image data processing method for the automatic parking scene provided in the embodiment of the present application obtains the current frame image data, and the current frame image data is not the first frame image data. The current frame image data and the previous frame image data corresponding to the current frame image data are interpolated based on the preset interpolation algorithm to obtain the interpolated frame image data between the current frame image data and the previous frame image data. When the current frame image data is the second frame image data, the interpolated frame image data is drawn based on the preset delay time, and the preset delay time is the generation time of each frame image data of the parked vehicle. When the current frame image data is not the second frame image data, the interpolated frame image data and the current frame image data are drawn at a fixed time interval. The embodiment of the present application uses a preset interpolation algorithm in the automatic parking scene to interpolate between the current frame and the previous frame image data, so as to improve the parking picture generated per second during automatic parking, and can solve serious problems such as unsmooth parking pictures and freezes, so that the automatic parking rendering is smoother and more realistic, and the user experience is improved.

[0113] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0114] Reference Figure 4 , shows a schematic diagram of the structure of an image data processing device for an automatic parking scene provided by an embodiment of the present application. Figure 4 As shown, the image data processing device 400 for the automatic parking scene may include the following modules:

[0115] The current frame data acquisition module 410 is used to acquire the current frame image data, where the current frame image data is not the first frame image data;

[0116] The interpolation frame data acquisition module 420 is used to perform interpolation processing on the current frame image data and the previous frame image data corresponding to the current frame image data based on a preset interpolation algorithm to obtain interpolation frame image data between the current frame image data and the previous frame image data;

[0117] The interpolation frame data processing module 430 is used to perform image drawing on the interpolation frame image data based on a preset delay time when the current frame image data is the second frame image data, wherein the preset delay time is the generation time of each frame image data of the parked vehicle;

[0118] The image data processing module 440 is used to perform image rendering on the interpolated frame image data and the current frame image data at a fixed time interval when the current frame image data is not the second frame image data.

[0119] Optionally, the interpolation frame data acquisition module includes:

[0120] An image frame number determination unit, used to determine the number of image frames to be interpolated between two adjacent frames of image data according to a preset required number of images per second and an image frame rate of the vehicle;

[0121] The interpolation frame image data acquisition unit is used to perform interpolation processing on the current frame image data and the previous frame image data by using a linear interpolation algorithm to obtain the interpolation frame image data of the image frame quantity.

[0122] Optionally, the interpolation image data acquisition unit includes:

[0123] a parameter acquisition subunit, configured to acquire, according to parking requirement information, a first parameter required for image drawing in the previous frame of image data and a second parameter matching the first parameter in the current frame of image data;

[0124] The interpolation frame data acquisition subunit is used to use the linear interpolation algorithm to perform interpolation processing on the current frame image data and the previous frame image data according to the first parameter and the second parameter to obtain the interpolation frame image data.

[0125] Optionally, when the first parameter and the second parameter are parameters of a parking space, the first parameter is four parking space corner points of the parking space in the previous frame image data, and the second parameter is four parking space corner points of the parking space in the current frame image data; the device further includes:

[0126] A first distance acquisition module, configured to acquire a first interval distance between the first parameter and the corresponding second parameter when the current frame image data is not the first frame image data and the first parameter and the second parameter are parameters of a parking space;

[0127] The first image drawing module is used for not performing interpolation processing on the current frame image data and using the current frame image data as the first frame image data for image drawing when the first interval distance is greater than a first distance threshold.

[0128] Optionally, when the first parameter and the second parameter are parameters of an obstacle, the first parameter is a center point of the obstacle in the previous frame of image data and a curvature parameter of the obstacle relative to the vehicle, and the second parameter is a center point of the obstacle in the current frame of image data and a curvature parameter of the obstacle relative to the vehicle; the device further includes:

[0129] a second distance acquisition module, configured to acquire, when the current frame image data is not the first frame image data and the first parameter and the second parameter are parameters of the obstacle, a second interval distance between a center point of the obstacle in the current frame image data and a center point of the obstacle in the previous frame image data, and an absolute value of a radian change parameter between a radian parameter corresponding to the current frame image data and a radian parameter corresponding to the previous frame image data;

[0130] The second image drawing module is used to not perform interpolation processing on the current frame image data and use the current frame image data as the first frame image data for image drawing when the second interval distance is greater than the second interval distance threshold and / or the absolute value of the curvature change parameter is greater than the curvature parameter threshold.

[0131] Optionally, the interpolation frame data acquisition module includes:

[0132] An image frame position reserving unit, used for reserving an image frame position of interpolated frame data in a pre-established cache queue, and writing the current frame image data into a position after the image frame position in the cache queue;

[0133] An image data acquisition unit, configured to perform interpolation processing on the current frame image data and the previous frame image data based on the preset interpolation algorithm to obtain the interpolation frame image data;

[0134] The interpolation frame data writing unit is used to write the interpolation frame image data into the image frame position in the cache queue.

[0135] Optionally, the interpolation data processing module includes:

[0136] A delay time acquisition unit, used for acquiring the interpolation frame data processing time corresponding to the interpolation frame image data, and using the interpolation frame data processing time as the preset delay time;

[0137] The image drawing unit is used to start timing from the moment when the current frame image data is received, and after delaying the preset delay time, call a pre-established thread to read the interpolated frame image data from the cache queue and perform image drawing.

[0138] Optionally, the image data processing module includes:

[0139] A time interval determination unit, used to determine the fixed time interval according to a preset number of required images per second;

[0140] The image data reading and drawing unit is used to call a pre-established thread to read the image data from the cache queue and perform image drawing.

[0141] Optionally, the device further comprises:

[0142] A frame number acquisition module, used to acquire the number of image frames of the image data cached in the cache queue;

[0143] A first data discarding module, configured to discard the earliest frame of image data cached in the cache queue when the number of image frames is greater than a first multiple of the interpolated frame image data cached in the cache queue;

[0144] A second data discarding module, configured to discard N frames of image data with the earliest cache time in the cache queue when the number of image frames is greater than a second multiple of the interpolated frame image data cached in the cache queue, where N is a positive integer and is half of the number of image frames;

[0145] The second multiple is greater than the first multiple.

[0146] The image data processing device for the automatic parking scene provided in the embodiment of the present application obtains the current frame image data, which is not the first frame image data. The current frame image data and the previous frame image data corresponding to the current frame image data are interpolated based on the preset interpolation algorithm to obtain the interpolated frame image data between the current frame image data and the previous frame image data. When the current frame image data is the second frame image data, the interpolated frame image data is drawn based on the preset delay time, and the preset delay time is the generation time of each frame image data of the parked vehicle. When the current frame image data is not the second frame image data, the interpolated frame image data and the current frame image data are drawn at a fixed time interval. The embodiment of the present application uses a preset interpolation algorithm in the automatic parking scene to interpolate between the current frame and the previous frame image data, thereby improving the parking picture generated per second during automatic parking, and can solve serious problems such as unsmooth parking pictures and freezes, so that the automatic parking rendering is smoother and more realistic, and improves the user experience.

[0147] It should be noted that the device provided in the above embodiment, when implementing its functions, is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0148] Please refer to Figure 5, which shows a block diagram of a computer device provided in one embodiment of the present application. The computer device can be used to implement the image data processing method of the automatic parking scene provided in the above embodiment. The computer device can be a PC or a server, or other devices with data processing and storage capabilities. Specifically:

[0149] The computer device 500 includes a central processing unit (CPU) 501, a system memory 504 including a random access memory (RAM) 502 and a read-only memory (ROM) 503, and a system bus 505 connecting the system memory 504 and the central processing unit 501. The computer device 500 also includes a basic input / output system (I / O system) 506 that helps transfer information between various components in the computer, and a large-capacity storage device 507 for storing an operating system 513, application programs 514, and other program modules 515.

[0150] The basic input / output system 506 includes a display 508 for displaying information and an input device 509 such as a mouse and a keyboard for user inputting information. The display 508 and the input device 509 are connected to the central processing unit 501 through an input / output controller 510 connected to the system bus 505. The basic input / output system 506 may also include an input / output controller 510 for receiving and processing inputs from a plurality of other devices such as a keyboard, a mouse, or an electronic stylus. Similarly, the input / output controller 510 also provides output to a display screen, a printer, or other types of output devices.

[0151] The mass storage device 507 is connected to the central processing unit 501 through a mass storage controller (not shown) connected to the system bus 505. The mass storage device 507 and its associated computer readable media provide non-volatile storage for the computer device 500. That is, the mass storage device 507 may include a computer readable medium (not shown) such as a hard disk or a CD-ROM drive.

[0152] Without loss of generality, the computer-readable medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules or other data. Computer storage media include RAM, ROM, EPROM, EEPROM, flash memory or other solid-state memory technology, CD-ROM, DVD or other optical storage, tape cassettes, magnetic tapes, disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that the computer storage media is not limited to the above. The above-mentioned system memory 504 and mass storage device 507 can be collectively referred to as memory.

[0153] According to various embodiments of the present application, the computer device 500 can also be connected to a remote computer on the network through a network such as the Internet. That is, the computer device 500 can be connected to the network 512 through the network interface unit 511 connected to the system bus 505, or the network interface unit 511 can be used to connect to other types of networks or remote computer systems (not shown).

[0154] The memory further includes one or more programs, which are stored in the memory and configured to be executed by one or more processors. The one or more programs include instructions for executing the image data processing method for the automatic parking scene.

[0155] In an example embodiment, a computer device is also provided, the computer device comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, code set or instruction set. The at least one instruction, at least one program, code set or instruction set is configured to be executed by one or more processors to implement the above-mentioned image data processing method for the automatic parking scene.

[0156] In an exemplary embodiment, a computer-readable storage medium is also provided, in which at least one instruction, at least one program, a code set or an instruction set is stored. When the at least one instruction, the at least one program, the code set or the instruction set is executed by a processor of a computer device, the image data processing method for the above-mentioned automatic parking scenario is implemented.

[0157] Optionally, the computer-readable storage medium may be a ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

[0158] In an exemplary embodiment, a computer program product is also provided. When the computer program product is executed, it is used to implement the image data processing method of the automatic parking scene described above.

[0159] It should be understood that the "plurality" mentioned in this article refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0160] The above description is only an exemplary embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for processing image data of an automatic parking scene, characterized in that: The method comprises: Acquire current frame image data, where the current frame image data is not the first frame image data; Performing interpolation processing on the current frame image data and the previous frame image data corresponding to the current frame image data based on a preset interpolation algorithm to obtain interpolation frame image data between the current frame image data and the previous frame image data; In a case where the current frame image data is the second frame image data, image rendering is performed on the interpolated frame image data based on a preset delay time, wherein the preset delay time is a generation time of each frame image data of the parked vehicle; In a case where the current frame image data is not the second frame image data, image rendering is performed on the interpolated frame image data and the current frame image data at a fixed time interval.

2. The method according to claim 1, characterized in that The interpolation processing is performed on the current frame image data and the previous frame image data corresponding to the current frame image data based on a preset interpolation algorithm to obtain the interpolation frame image data between the current frame image data and the previous frame image data, including: Determining the number of image frames to be interpolated between two adjacent frames of image data according to a preset number of required images per second and an image frame rate of the vehicle; A linear interpolation algorithm is used to perform interpolation processing on the current frame image data and the previous frame image data to obtain the interpolation frame image data of the image frame quantity.

3. The method according to claim 2, characterized in that The interpolation process of the current frame image data and the previous frame image data using a linear interpolation algorithm to obtain the interpolation frame image data of the number of image frames includes: According to the parking requirement information, obtaining a first parameter required for image drawing in the previous frame of image data and a second parameter matching the first parameter in the current frame of image data; The linear interpolation algorithm is used to perform interpolation processing on the current frame image data and the previous frame image data according to the first parameter and the second parameter to obtain the interpolation frame image data.

4. The method according to claim 3, characterized in that When the first parameter and the second parameter are parameters of a parking space, the first parameter is four parking space corner points of the parking space in the previous frame image data, and the second parameter is four parking space corner points of the parking space in the current frame image data; After obtaining the current frame image data, the method further includes: Obtaining a first interval distance between the first parameter and the corresponding second parameter; When the first interval distance is greater than a first distance threshold, no interpolation processing is performed on the current frame image data, and the current frame image data is used as the first frame image data for image drawing.

5. The method according to claim 3, characterized in that: When the first parameter and the second parameter are parameters of an obstacle, the first parameter is a center point of the obstacle in the previous frame of image data and a curvature parameter of the obstacle relative to the vehicle, and the second parameter is a center point of the obstacle in the current frame of image data and a curvature parameter of the obstacle relative to the vehicle; After obtaining the current frame image data, the method further includes: Acquire a second interval distance between the obstacle center point in the current frame image data and the obstacle center point in the previous frame image data, and an absolute value of a radian change parameter between the radian parameter corresponding to the current frame image data and the radian parameter corresponding to the previous frame image data; When the second spacing distance is greater than the second spacing distance threshold, and / or the absolute value of the curvature change parameter is greater than the curvature parameter threshold, the current frame image data is not interpolated, and the current frame image data is used as the first frame image data for image drawing.

6. The method according to any one of claims 1 to 5, characterized in that: The interpolation processing is performed on the current frame image data and the previous frame image data corresponding to the current frame image data based on a preset interpolation algorithm to obtain the interpolation frame image data between the current frame image data and the previous frame image data, including: Reserving an image frame position of the interpolated frame data in a pre-established cache queue, and writing the current frame image data into a position after the image frame position in the cache queue; Performing interpolation processing on the current frame image data and the previous frame image data based on the preset interpolation algorithm to obtain the interpolation frame image data; The interpolated frame image data is written into the image frame position in the cache queue.

7. The method according to claim 6, characterized in that The performing image drawing on the interpolated frame image data based on the preset delay time includes: Acquire the interpolation frame data processing time corresponding to the interpolation frame image data, and use the interpolation frame data processing time as the preset delay time; Timing starts from the moment when the current frame image data is received, and after delaying the preset delay time, a pre-established thread is called to read the interpolated frame image data from the cache queue and perform image drawing.

8. The method according to claim 6, characterized in that The performing image drawing on the interpolated frame image data and the current frame image data at a fixed time interval includes: Determining the fixed time interval according to a preset number of required images per second; A pre-established thread is called to read image data from the cache queue and perform image drawing.

9. The method according to claim 6, characterized in that The method further comprises: Obtain the number of image frames of image data cached in the cache queue; When the number of image frames is greater than a first multiple of the interpolated frame image data cached in the cache queue, discarding the earliest frame of image data cached in the cache queue; When the number of image frames is greater than the second multiple of the interpolated frame image data cached in the cache queue, discarding the N frames of image data cached earliest in the cache queue, where N is a positive integer and is half of the number of image frames; The second multiple is greater than the first multiple.

10. An image data processing device for an automatic parking scene, characterized in that: The device comprises: A current frame data acquisition module, used to acquire current frame image data, where the current frame image data is not the first frame image data; An interpolation frame data acquisition module, configured to perform interpolation processing on the current frame image data and the previous frame image data corresponding to the current frame image data based on a preset interpolation frame algorithm to obtain interpolation frame image data between the current frame image data and the previous frame image data; an interpolation frame data processing module, configured to, when the current frame image data is the second frame image data, perform image drawing on the interpolation frame image data based on a preset delay time, wherein the preset delay time is a generation time of each frame image data of the parked vehicle; The image data processing module is used to draw images on the interpolated frame image data and the current frame image data at a fixed time interval when the current frame image data is not the second frame image data.

11. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method according to any one of claims 1 to 10.

12. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method according to any one of claims 1 to 10.

Images