Image data processing method, device, medium, product and vehicle

Abstract

The present invention relates to the field of data processing technology, and in particular to an image data processing method, device, medium, product and vehicle. The method comprises: receiving frame data captured by a camera, deleting non-continuous frame data, and outputting a target image frame. The present invention can ensure the continuity of frame data of a spliced ​​image frame, improve imaging clarity, and ensure the display effect of the target image frame.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to an image data processing method, device, medium, product and vehicle. Background Art

[0002] Existing cameras usually directly output odd frame data and even frame data alternately and stitch them together. This may result in discontinuous frame data output, damaged or lost frame data, resulting in discontinuous spliced ​​image frames or screen distortion, blur, etc., affecting image clarity. Summary of the invention

[0003] The present invention aims at the technical problems such as poor imaging clarity of cameras in the prior art and provides an image data processing method, device, medium, product and vehicle.

[0004] The invention provides an image data processing method, comprising: receiving frame data shot by a camera, deleting non-continuous frame data, and outputting a target image frame.

[0005] The present invention also provides an electronic device, comprising a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to execute the computer program to implement the above-mentioned image data processing method.

[0006] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions implement the above-mentioned image data processing method when executed by a processor.

[0007] The present invention also provides a computer program product, comprising a computer program, wherein the computer program implements the above-mentioned image data processing method when executed by a processor.

[0008] The present invention also provides a vehicle, comprising the above electronic device or the above computer program product.

[0009] In the image data processing method, device, medium, product and vehicle provided by the present invention, the method comprises: receiving frame data captured by a camera, deleting non-continuous frame data, and outputting a target image frame. In the present invention, the frame data captured by the camera is processed, non-continuous frame data is dropped, and the frame data of the spliced ​​image frame is ensured to be continuous; the occurrence of screen distortion, blurring and the like caused by the splicing of non-continuous frame data is avoided, the imaging clarity is improved, and the display effect of the target image frame is ensured. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0011] Figure 1 is a flow chart of an image data processing method in one embodiment of the present invention;

[0012] Figure 2 is a principle block diagram of an image data processing device in one embodiment of the present invention;

[0013] Figure 3 is a schematic diagram of an electronic device in one embodiment of the present invention. DETAILED DESCRIPTION

[0014] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0015] like Figure 1 As shown, the present invention provides an image data processing method, comprising:

[0016] S10, receiving frame data captured by a camera, deleting non-continuous frame data, and outputting a target image frame.

[0017] In the above-mentioned embodiment of the present invention, the frame data captured by the camera is processed, and non-continuous frame data are discarded to ensure the continuity of the frame data of the spliced ​​image frame; the occurrence of screen noise, blur, etc. caused by the splicing of non-continuous frame data is avoided, the imaging clarity is improved, and the display effect of the target image frame is guaranteed.

[0018] Furthermore, the image data processing method is applied to a system on chip (SOC). In the present invention, the installation object of the system on chip SOC is not set. The SOC can be a SOC in a vehicle's on-board system for controlling a vehicle camera, or a SOC in a traffic system for managing traffic electronic eyes. The SOC can also be a SOC in an electronic product for controlling a camera, etc.; as long as the installation object of the SOC is equipped with a camera and the SOC needs to be used to execute the image data processing method of the present invention to ensure the image display effect.

[0019] In one embodiment, the frame data includes image frame data and image frame data identifiers, and the image frame data identifiers include odd frames and even frames. It is understandable that the camera in the present invention can be an odd-even camera, so the frame data includes image frame data captured by the odd-even camera. It is understandable that since the cost of the odd-even frame camera is lower, the cost can be reduced by capturing frame data by the odd-even camera (the frame data of each frame is one of the frames of the video data captured continuously by the odd-even camera). The image frame data captured by the odd-even camera is output in an interlaced manner, that is, the image frame data includes odd frame data (frame data of odd frames) and even frame data (frame data of even frames), wherein the odd frame data includes 1, 3, 5, 7... and other multiple odd frame lines captured by the odd-even camera at the same time point; and the even frame data includes 0, 2, 4, 6... and other multiple even frame lines captured by the odd-even camera at the same time point.

[0020] Specifically, after the preset application sends a shooting request to the SOC, the SOC will instruct the odd-even camera to shoot the image frame data. Afterwards, when the odd-even camera shoots the image frame data and outputs it, it will mark each image frame data with an image frame data identifier. Therefore, the above frame data also includes an image frame data identifier associated with each image frame data. Then, the image frame data identifier can be used to determine whether the corresponding image frame data is odd frame data or even frame data. It is understandable that the image frame data identifier can be set according to demand, as long as it can be used to distinguish between odd frame data and even frame data. In one embodiment, the image frame data identifier includes odd frames and even frames, and the odd frames and even frames can be respectively set to 0 and 1. It is understandable that the odd frames and even frames can be set to other identifiers, which are not limited here.

[0021] In one embodiment, in step S10, the receiving of frame data captured by a camera includes: deserializing the image frame data, receiving the deserialized image frame data through a first preset interface, and receiving the image frame data identifier through a second preset interface; the first preset interface and the second preset interface are different types of interfaces; and the image data processing method is applied to a system on chip SOC.

[0022] Furthermore, the first preset interface is a mobile industry processor serial interface (Mobile Industry Processor Interface-Camera Serial Interface2, mipi-csi2). The second preset interface is a general-purpose input / output interface (General-purpose input / output, gpio).

[0023] Specifically, the core layer of the SOC is used to receive image frame data. It is understandable that the third interface is connected between the camera (such as an odd-even camera) and the serializer, the second interface is connected between the serializer and the deserializer, and the first interface is set in the core layer of the SOC and is used to connect the end of the deserializer away from the serializer. Further, the first interface, the second interface and the third interface are all serial interfaces of mobile industry processors. The first preset interface includes but is not limited to the first interface; in one embodiment, specifically, the core layer of the SOC receives the deserialized image frame data sent by the deserializer through the first interface; the deserialized image frame data refers to the image frame data obtained by the deserializer after the deserializer receives the serial signal sent by the serializer through the second interface; the serial signal refers to the image frame data captured by the camera (i.e., the odd-even camera) and sent to the serializer through the third interface, and the serializer performs string processing on the image frame data to obtain the signal.

[0024] It can be understood that when the camera is an odd-even camera, after the odd-even camera captures the image frame data, the image frame data will first be sent to the serializer through the third interface, and after the serializer performs serialization processing on the image frame data, the image frame data will be converted into a serial signal and then passed to the deserializer through the second interface. Then, the deserializer deserializes the serial signal after receiving it, and then obtains the deserialized image frame data corresponding to the above image frame data; it can be understood that the data transmission protocol used between the serializer and the deserializer is the composite synchronous video broadcast signal (Composite Video Broadcast Signal, cvbs), Gigabit Multimedia Serial Link (Gigabit Multimedia Serial Link, gmsl) and the field bus communication protocol (Flat Panel Display Link, fpd-link) to ensure long-distance and stable transmission of data. Finally, the deserializer sends the deserialized image frame data to the core layer of the SOC, so as to receive the deserialized image frame data through the first interface of the core layer of the SOC; in this way, the image frame data is transmitted from the odd-even camera to the core layer of the SOC through multiple mobile industry processor serial interfaces, and the odd-even frame data is transmitted through the mobile industry processor serial interface, serializer, deserializer and other hardware, with strong anti-interference ability, so that the risk of loss in the data transmission process is relatively small, and the data transmission process is more stable and reliable.

[0025] In this embodiment, while receiving the image frame data captured by the camera, the core layer of the SOC in this embodiment also synchronously receives the image frame data identifier in the frame data. Specifically, the odd-even camera maps the image frame data identifier corresponding to the image frame data to the serializer through the first port, and the serializer maps the image frame data identifier to the deserializer through the second port, and then the third port built into the SOC synchronously obtains the image frame data identifier from the deserializer.

[0026] Understandably, the first port is connected between the odd-even camera and the serializer, the second port is connected between the serializer and the deserializer, and the third port is arranged at the core layer of the SOC and is used to connect the end of the deserializer away from the serializer. Among them, the first port, the second port and the third port are all general input and output ports. The above mapping method refers to the gpio mapping method. The second preset interface includes but is not limited to the third port. Therefore, in this embodiment, after the odd-even camera shoots the image frame data and marks the image frame data identifier for each image frame data, the image frame data is transmitted to the core layer of the SOC through the mobile industry processor serial interface, the serializer, the deserializer, etc. in the above embodiment, and the image frame data identifier is first transmitted to the serializer through the first port in a mapping manner. After that, the serializer transmits the image frame data identifier to the deserializer through the second port in a mapping manner, and the third port built into the SOC receives the image frame data identifier transmitted by the deserializer in a mapping manner through the third port. In this way, the image frame data identification uses the hardware port (i.e., the first port, the second port, and the third port) to map the image frame data identification to the core layer of the SOC. The transmission process is stable, safe, reliable, and concise, and the operation process is simple and easy.

[0027] It can be understood that the SOC has multiple ports built in (all of the multiple ports can be general input and output ports, and can also include other types of ports). In the process of mapping the image frame data identifier from the decoder to the core layer of the SOC, the SOC only needs to use a third port among the multiple ports (the third port is a general input and output port) to synchronously receive the image frame data identifier in the deserializer (there is no need to use multiple ports to receive the image frame data identifier, thereby avoiding waste of port resources).

[0028] In this embodiment, the odd-even camera outputs the image frame data identifier and the image frame data through the first preset interface and the second preset interface respectively, and the first line of each image frame data is normal data, and there is no need to set the first line of the even frame data (or even frame data) as a blank line to distinguish between odd frame data and even frame data. In this way, the image frame data will not be occupied by the image frame data identifier (that is, the blank line) to occupy the data transmission bandwidth and affect the data quality. Therefore, in this embodiment, the image frame data and the image frame data identifier are transmitted separately to save the transmission bandwidth of the image frame data and ensure data quality.

[0029] In the above embodiment of the present invention, the image frame data is transmitted to the core layer of the SOC through multiple mobile industry processor serial interfaces, serializers, deserializers and other hardware, and the image frame data identifier is transmitted to the core layer of the SOC through multiple ports, serializers, deserializers and other hardware. In the above transmission process, the image frame data and the image frame data identifier are respectively transmitted through different hardware channels. In this way, not only can the transmission bandwidth of the image frame data be saved and the data quality be guaranteed (the image frame data will not be occupied by the image frame data identifier and occupy the data transmission bandwidth and affect the data quality), but the method of realizing data transmission through hardware can also reduce the risk of data loss, ensure that the data transmission process is more stable and reliable, and ensure the accuracy of signal transmission at the hardware level.

[0030] In one embodiment, in the step S10, after the frame data captured by the camera is received, the step includes: marking the system receiving time as the time information of the frame data.

[0031] That is, in this embodiment, it is necessary to mark each of the image frame data with time information corresponding to the system receiving time, and the time information can be represented in the form of a timestamp. In the present invention, the core layer of the SOC can also mark the image frame data with time information corresponding to the system receiving time according to the system receiving time when the SOC receives the image frame data, while receiving the image frame data and the image frame data identifier.

[0032] In one embodiment, marking time information on the frame data includes:

[0033] At the system receiving time when the core layer of the SOC receives the image frame data, the identification status of the received image frame data identifier is read; that is, when the core layer of the SOC receives the image frame data identifier, the identification status corresponding to the image frame data identifier can be synchronously read. In one embodiment, the identification status includes a low level state and a high level state, wherein the low level state represents that the image frame data identifier is an odd frame (representing that the image frame data corresponding to the odd frame is odd frame data), and the high level state represents that the image frame data identifier is an even frame (representing that the image frame data corresponding to the even frame is even frame data).

[0034] The core layer marks the identification status and the time information corresponding to the system receiving time in the structure of the image frame data, and sends the image frame data marked with the identification status and the time information to the hardware abstraction layer. That is, after the core layer of the SOC receives the image frame data and the image frame data identification, the system receiving time when the SOC receives the image frame data can be determined. After that, the identification status read in the previous step and the time information corresponding to the system receiving time are marked together in the structure of the image frame data. In this way, the structure of the image frame data contains the identification status corresponding to the image frame data identification and the time information corresponding to the system receiving time, so that in the subsequent steps, each image frame data can be spliced ​​according to the time information and image frame data identification marked in each image frame data, and then the target image frame is output.

[0035] Furthermore, the core layer marks the identification state and the time information corresponding to the system receiving time in the structure of the image frame data, including: the core layer adds a marking variable in the structure of the image frame data, and writes the identification state and the time information corresponding to the system receiving time into the identification variable of the image frame data, so as to mark the image frame data identification and the time information in the structure of the image frame data. That is, in this embodiment, each image frame data has a data structure corresponding to it, as long as the marking variables corresponding to the identification state and the time information are added to the data structure corresponding to the image frame data according to the needs, the identification state and the time information corresponding to the image frame data can be recorded in the marking variables in the data structure, thereby obtaining the structure of the image frame data marked with the image frame data identification (that is, the identification state) and the time information (corresponding to the system receiving time). Afterwards, the image frame data marked with the image frame data identifier and the time information will be passed from the core layer of the SOC to the hardware abstraction layer of the SOC. The hardware abstraction layer of the SOC is a connecting layer. The upper layer needs to send a request to the hardware abstraction layer for data, and the hardware abstraction layer returns the data obtained by the bottom layer to the upper layer.

[0036] In one embodiment, the non-continuous frame data includes: the difference in time information of two adjacent frame data is greater than a preset time threshold; and / or the two adjacent frame data do not include an odd frame and an even frame. Specifically, when the two adjacent frame data do not include an odd frame and an even frame, it means that the two adjacent frame data do not meet the parity judgment; and when the image frame data identifiers of the two adjacent frame data include an odd frame and an even frame, it is considered that the two adjacent frame data meet the parity judgment. Specifically, the frame data can be judged according to the above two conditions, that is, when the difference in time information of the two adjacent frame data is greater than the preset time threshold, the two adjacent frame data are confirmed to be non-continuous frame data; and when the two adjacent frame data do not include an odd frame and an even frame, it means that the two adjacent frame data do not meet the parity judgment, and the two adjacent frame data can also be confirmed to be non-continuous frame data. Among them, the preset time threshold can be set according to demand, for example, the preset time threshold can be set to 40ms.

[0037] Further, in step S10, deleting the non-continuous frame data includes: after determining the non-continuous frame data, deleting the non-continuous frame data. Wherein, after determining the non-continuous frame data, deleting the non-continuous frame data specifically includes: determining two frames of the image frame data that are adjacent in time when received by the system as a comparison frame group, and after determining that the comparison frame group is non-continuous frame data according to the time information and the image frame data identifier, deleting the non-continuous frame data. Specifically, when the hardware abstraction layer of the SOC determines that the image frame data of the two frames in the comparison frame group are non-continuous frame data, it means that abnormal situations such as bad frames and data abnormalities have occurred at this time, and the contents of the two frames of image frame data in the comparison frame group are quite different. Therefore, the two frames of image frame data in the comparison frame group will not be spliced. Instead, it is necessary to use frame drop processing for the image frame data in the comparison frame group that is received by the system earlier, so that the comparison frame groups with large differences will not be forcibly spliced ​​and affect the quality of the target image frames obtained, so as to ensure the accuracy of the content of the spliced ​​comparison frame group and solve the influence of abnormal situations on the display effect of the output image.

[0038] It is understandable that in this step, the time information and image frame data identifier (i.e., the identifier status marked in the image frame data) marked in the image frame data are used to determine whether each comparison frame group is non-continuous frame data. In this embodiment, non-continuous frame data mainly includes two conditions: "non-continuous" and "judgment if not satisfied". Specifically, two frames of image frame data in the same comparison frame group are not image frame data continuously shot by an odd-even camera within a preset time threshold, and / or two frames of image frame data in the comparison frame group do not meet the condition that one of them is an odd frame and the other is an even frame, and then the two adjacent frame data in the comparison frame group are considered to be non-continuous frame data. In this embodiment, by judging whether the comparison frame group is non-continuous frame data through the above process, and then deleting the non-continuous frame data, the frame data that can be spliced ​​is determined, and then the accuracy of splicing can be improved, so that the display effect of the output target image frame is better.

[0039] Understandably, in this embodiment, each comparison frame group is composed of the image frame data of the two frames before and after which the system receives the data at adjacent times. Taking the odd-even camera of 30fps odd frame and 30fps even frame as an example, the odd-even camera outputs 30fps odd frame and 30fps even frame, which are sorted in order according to the time information: odd frame 0, even frame 0, odd frame 1, even frame 1, odd frame 2, even frame 2... At this time, odd frame 0 and even frame 0 will form a comparison frame group, even frame 0 and odd frame 1 will form a comparison frame group, odd frame 1 and even frame 1 will form a comparison frame group, even frame 1 and odd frame 2 will form a comparison frame group, odd frame 2 and even frame 2 will form a comparison frame group... By analogy, 30 odd frames and 30 even frames will form 60 comparison frame groups (instead of 30 groups). Furthermore, when the above comparison frame groups are not non-continuous frame data, all comparison frame groups can be fused and spliced ​​to achieve 60fps output, thereby improving the frame rate of SOC. When some of the comparison frame groups are non-continuous frame data, the non-continuous frame data will be deleted and then the remaining comparison frame groups will be spliced.

[0040] In one embodiment, determining that the comparison frame group is non-continuous frame data according to the time information and the image frame data identifier includes:

[0041] When the hardware abstraction layer of the SOC determines that the image frame data identifiers corresponding to the two frames of the image frame data in the comparison frame group are the same, or when it is determined based on the time information that the difference in time information corresponding to the two frames of the image frame data in the comparison frame group is greater than a preset time threshold, it is confirmed that the two frames of image data in the comparison frame group are non-continuous frame data. Wherein, the preset time threshold can be set according to demand, for example, the preset time threshold can be set to 40ms, so that when the difference in time information corresponding to the two frames of the image frame data in the comparison frame group is greater than the preset time threshold, it is considered that the two frames of image frame data in the same comparison frame group are both image frame data taken by odd-even cameras, but the two frames of image frame data are not taken continuously, so the two frames of image frame data in the comparison frame group are non-continuous frame data. When it is determined that the image frame data identifiers corresponding to the two frames of the image frame data in the comparison frame group are the same, it can be confirmed that the two frames of image frame data in the same comparison frame group are both odd frames or both even frames, and at this time, the two frames of image frame data in the comparison frame group will also be confirmed as non-continuous frame data. It can be understood that when two frames of image frame data in a comparison frame group are confirmed as non-continuous frame data, it means that the contents of the two frames of image frame data in the comparison frame group are greatly different, and the two frames of image frame data in the comparison frame group are not allowed to be spliced ​​in subsequent steps. Instead, it is necessary to use frame drop processing for one of the image frame data that is received by the system earlier, so that the comparison frame groups with large differences will not be forcibly spliced ​​to affect the quality of the obtained target image frames, so as to ensure the accuracy of the content of the spliced ​​comparison frame groups.

[0042] Further, when the hardware abstraction layer of the SOC determines that the image frame data identifiers corresponding to the two frames of the image frame data in the comparison frame group are not the same, and determines according to the time information that the difference in time information corresponding to the two frames of the image frame data in the comparison frame group is less than or equal to the preset time threshold, it is confirmed that the two frames of image data in the comparison frame group are not discontinuous frame data. Understandably, when it is determined that the difference in time information corresponding to the two frames of the image frame data in the comparison frame group is less than or equal to the preset time threshold (indicating that the two frames of image frame data in the same comparison frame group are image frame data continuously shot by odd-even cameras), and the image frame data identifiers corresponding to the two frames of the image frame data in the comparison frame group are not the same (indicating that one of the two frames of image frame data in the same comparison frame group is an odd frame and the other is an even frame), the hardware abstraction layer can confirm that the two frames of image frame data in the comparison frame group are not discontinuous frame data, but continuous frame data, and at this time, the two frames of image frame data in the comparison frame group will be allowed to be spliced ​​in subsequent steps.

[0043] Understandably, since bad frames and data anomalies are inevitable in the transmission process of image frame data (bad frames and data anomalies refer to temporary data anomalies caused by hardware interference and lighting conditions of odd-even cameras, etc.), at this time, if the above-mentioned method in this embodiment is not used to use time information and image frame data identification to judge whether the image frame data in the comparison frame group is non-continuous frame data, but all comparison frame groups are directly spliced, at this time, due to the existence of bad frames and data anomalies, etc., there will be different situations such as odd-even splicing, odd-odd splicing, even-even splicing, and even-odd splicing in the splicing of the comparison frame groups, resulting in data fusion failure in the subsequent splicing process, large differences in splicing content, etc., which will cause the target image frame output after splicing to appear sometimes clear and sometimes blurred, and jump up and down, etc. Abnormal phenomenon, which will cause customer complaints. Therefore, by using the present embodiment to determine whether the image frame data in the comparison frame group is non-continuous frame data based on the time information and the image frame data identifier, and then determining whether the two frames of image frame data in the comparison frame group can be spliced ​​in subsequent steps, the accuracy of splicing can be improved, so that the display effect of the output target image frame is better.

[0044] In one embodiment, in step S10, the output of the target image frame includes: splicing the frame data of the odd frame with the frame data of the even frame at the previous moment and / or the frame data of the even frame at the next moment to obtain the target image frame and output it. In this embodiment, after deleting the non-continuous frame data, the two frames of the image frame data in each remaining comparison frame group can be spliced ​​into a target image frame and output, specifically, the frame data of each remaining odd frame is spliced ​​with the frame data of the even frame at the previous moment or / and the next moment to obtain the target image frame and output it. It can be understood that the hardware abstraction layer of the SOC will fill the two frames of image frame data in the comparison frame group remaining after deleting the non-continuous frame data into the target buffer, and output the filled target buffer to the preset application end that sends a shooting request to the SOC, and then display or save the target image frame at the preset application end. It can be understood that since the image frame data is one of the frames of video data continuously captured by the odd-even camera, when the odd-even camera continuously captures image frame data, the hardware abstraction layer of the SOC will also continuously output the target image frame, and then, the preset application end receives video data composed of multiple target image frames.

[0045] It is understandable that, in the case where the image frame data is output in interlaced format, if the image frame data captured by the odd-even camera is directly and continuously output without being processed, the image frame finally output will be half the size of the normal image frame, which will also cause the displayed image frame to have a jumping feeling and a relatively virtual picture quality. In the present embodiment, after deleting the non-continuous frame data, the remaining two frames of the image frame data in the comparison frame group are spliced ​​into the target image frame before being output, which can improve the resolution of each target image frame outputted and ensure the display effect of the output image.

[0046] In one embodiment, the frame data of the odd-numbered frame is spliced ​​with the frame data of the even-numbered frame at a previous moment and / or the frame data of the even-numbered frame at a subsequent moment to obtain and output a target image frame, including:

[0047] In the preset node of the hardware abstraction layer of the SOC, the odd frame rows and the even frame rows are alternately filled into the target buffer row by row; wherein, in the comparison frame group that meets the requirements of continuous odd and even frames, the two frames of the image frame data include an odd frame frame data and an even frame frame data; the odd frame row refers to each row of odd frame data in the frame data of the odd frame; the even frame row refers to each row of even frame data in the frame data of the even frame; that is, in this embodiment, the preset node includes but is not limited to a p2streaming node. Since the frame data of the odd frame includes 1, 3, 5, 7... and other multiple rows of odd frame rows taken by odd and even cameras at the same time point; and the frame data of the even frame includes 0, 2, 4, 6... and other multiple rows of even frame rows taken by odd and even cameras at the same time point. In this step, after the comparison frame group meets the requirements of continuous odd and even frames, the hardware abstraction layer reads the odd frame rows of the frame data of the odd frames in the comparison frame group and the even frame rows of the frame data of the even frames in sequence through the preset nodes, and then copies the odd frame rows and the even frame rows one by one in an alternating manner of one row of odd frame rows and one row of even frame rows and fills them into the target buffer.

[0048] After confirming that all the odd frame rows and the even frame rows of the comparison frame group are filled into the target buffer, all the odd frame rows and the even frame rows of the target buffer are merged in the preset node to generate a target image frame corresponding to the comparison frame group, and the target image frame is output to a preset application end for display.

[0049] Understandably, the hardware abstraction layer of the SOC will output the filled target buffer to the preset application end that sends a shooting request to the SOC, and then display or save the target image frame on the preset application end. Understandably, since the image frame data is one of the frames of the video data continuously shot by the odd-even camera, when the odd-even camera continuously shoots the image frame data, the hardware abstraction layer of the SOC will also continuously output the target image frame, and then, the preset application end receives video data composed of multiple target image frames, and at this time, the display form of the target image frame on the preset application end will be playing video.

[0050] In the above embodiment of the present invention, an odd-even camera is used to shoot image frame data, and the SOC uses time information and image frame data identifiers to determine the comparison frame group belonging to non-continuous frame data, and after deleting the non-continuous frame data, the two frames of image frame data in the remaining comparison frame group are spliced ​​and the target image frame is output. In this way, not only can the purpose of reducing costs be achieved by shooting with a lower-cost odd-even camera, but also the target image frame with high resolution can be spliced ​​through the above-mentioned image data processing method of the present invention to ensure the display effect of the target image frame. Furthermore, in the present invention, the SOC only splices continuous frame data into a target image frame, so that the two spliced ​​image frame data must be continuous and have opposite image frame data identifiers (that is, the two spliced ​​image frame data must be image frame data continuously shot by odd-even cameras, and one of them is frame data of an odd frame, and the other is frame data of an even frame), thereby avoiding the occurrence of screen distortion, blurring, etc. caused by the splicing between two non-continuous image frame data or between two image frame data with the same image frame data identifier, and further ensuring the display effect of the target image frame; therefore, the present invention can simultaneously achieve the purpose of reducing manufacturing costs and ensuring image display effects.

[0051] In one embodiment, the image frame data includes frame data of odd frames and frame data of even frames; the comparison frame group includes a first frame group and a second frame group; further, determining two frames of the image frame data received by the system at adjacent times as the comparison frame group includes:

[0052] After the hardware abstraction layer of the SOC receives the image frame data transmitted by the core layer, the hardware abstraction layer generates a frame sequence corresponding to all the image frame data according to the time information; all the image frame data in the frame sequence are sorted in the order of the system reception time corresponding to the time information; in this embodiment, in the frame sequence, the image frame data are arranged in order of the system reception time (it can be the earlier the system reception time, the earlier it is, or the later the system reception time, the earlier it is, which is not limited here), so as to facilitate the comparison of frame groups according to the frame sequence combination in the subsequent process.

[0053] The hardware abstraction layer determines the first comparison frame and the second comparison frame whose system reception time is adjacent in the frame sequence as the first frame group; the first comparison frame and the second comparison frame are the frame data of the odd frame or the frame data of the even frame in the image frame data; the system reception time of the first comparison frame is earlier than the system reception time of the second comparison frame; that is, taking an odd-even camera with 30fps odd frames and 30fps even frames as an example, the odd-even camera outputs 30fps odd frames and 30fps even frames, which are sorted from first to last according to the system reception time corresponding to the time information: odd frame 0, even frame 0, odd frame 1, even frame 1, odd frame 2, even frame 2... At this time, if odd frame 0 is the first comparison frame and even frame 0 is the second comparison frame, then odd frame 0 and even frame 0 will form the first frame group.

[0054] The hardware abstraction layer determines the second comparison frame and the third comparison frame whose system receiving time is adjacent in the frame sequence as the second frame group; the third comparison frame is the frame data of the odd frame or the frame data of the even frame in the image frame data; the system receiving time of the second comparison frame is earlier than the system receiving time of the third comparison frame. That is, taking the odd-even camera of 30fps odd frame and 30fps even frame as an example, the odd-even camera outputs 30fps odd frame and 30fps even frame, which are arranged from first to last according to the system receiving time corresponding to the time information: odd frame 0, even frame 0, odd frame 1, even frame 1, odd frame 2, even frame 2... At this time, if odd frame 0 is the first comparison frame and even frame 0 is the second comparison frame, then odd frame 1 is the third comparison frame. At this time, odd frame 0 and even frame 0 will form the first frame group, even frame 0 and odd frame 1 will form the second frame group..., and so on, 30 odd frames and 30 even frames will form 60 comparison frame groups (instead of 30 groups). Furthermore, when the above-mentioned comparison frame groups all meet the requirements of continuous odd-even frames, the comparison frame groups can be fused and spliced ​​after meeting the requirements of continuous odd-even frames, thereby achieving 60fps output, thereby improving the frame rate of the SOC.

[0055] In one embodiment, the target image frame includes a first target image frame and a second target image frame; further, in step S10, the outputting of the target image frame after deleting the non-continuous frame data includes:

[0056] When the first frame group and the second frame group are not non-continuous frame data, the hardware abstraction layer splices the first comparison frame and the second comparison frame in the first frame group into a first target image frame for output, and then splices the second comparison frame and the third comparison frame in the second frame group into a second target image frame for output. That is, in this embodiment, if the first frame group and the second frame group are not non-continuous frame data, therefore, there is no non-continuous frame data that needs to be deleted. At this time, the first frame group and the second frame group can be directly spliced. Then, after the hardware abstraction layer splices the first comparison frame and the second comparison frame in the first frame group into a first target image frame for output, the second comparison frame and the third comparison frame in the second frame group are spliced ​​into a second target image frame for output to the preset application end. At this time, the preset application end receives continuous video data composed of the first target image frame and the second target image frame.

[0057] In one embodiment, the target image frame includes a second target image frame; further, in step S10, the target image frame is output after deleting the non-continuous frame data, and further includes:

[0058] When the first frame group is non-continuous frame data, and the second frame group is not non-continuous frame data, the hardware abstraction layer performs frame dropping processing on the first comparison frame in the first frame group that is received earlier by the system, and then splices the second comparison frame and the third comparison frame in the second frame group into a second target image frame for output. That is, in this embodiment, if the first frame group is non-continuous frame data, and the second frame group is not non-continuous frame data, at this time, it is first necessary to perform frame dropping processing on the first comparison frame in the first frame group that is received earlier by the system, and then perform splicing processing on the second frame group. Then, after the hardware abstraction layer discards the first comparison frame, it splices the second comparison frame and the third comparison frame into a second target image frame and outputs it to the preset application end. At this time, the preset application end receives only the second target image frame.

[0059] In one embodiment, the target image frame includes a first target image frame; further, in step S10, the target image frame is output after deleting the non-continuous frame data, and further includes:

[0060] When the first frame group is not non-continuous frame data, and the second frame group is non-continuous frame data, the hardware abstraction layer splices the first comparison frame and the second comparison frame in the first frame group into a first target image frame for output, and then performs frame dropping processing on the second comparison frame in the second frame group that is received by the system earlier. That is, in this embodiment, if the first frame group is not non-continuous frame data, and the second frame group is non-continuous frame data, at this time, it is necessary to perform splicing processing on the first frame group, and then perform frame dropping processing on the second comparison frame in the second frame group that is received by the system earlier. Furthermore, after the hardware abstraction layer splices the first comparison frame and the second comparison frame into a first target image frame and outputs it to the preset application end, the second comparison frame is discarded. At this time, the preset application end receives only the first target image frame.

[0061] In one embodiment, the image data processing method further includes: when the target image frame is not output within a preset time interval, controlling the camera to restart. In this embodiment, the preset time interval can be set according to demand, that is, it is normal for the SOC to not output the target image frame within the preset time interval, and if the SOC still does not output the target image frame after exceeding the preset time interval (such as a few seconds), it means that the current response to the shooting request of the preset application end may be abnormal. At this time, the odd-even camera can be restarted to further ensure the stability of the response to the shooting request. In the present invention, the SOC splices the target image frame after deleting the non-continuous frame data. When there is a comparison frame group of non-continuous frame data, the splicing is not performed directly, but the image frame data received by the system earlier in the comparison frame group (that is, the comparison frame group of non-continuous frame data) is discarded to handle abnormal situations such as bad frames and data abnormalities, and solve the impact of abnormal situations on the display effect of the output image; and in this embodiment, combined with the solution of controlling the odd-even camera to restart when it is detected that the target image frame is not output within a preset time interval, the impact of hardware abnormalities such as odd-even cameras on the display effect of the output image can be further solved, and the output display effect is further guaranteed.

[0062] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0063] In one embodiment, if Figure 2 As shown, an image data processing device is provided, which corresponds one-to-one to the image data processing method in the above embodiment. The image data processing device includes:

[0064] The output module 100 is used to receive frame data captured by a camera, delete non-continuous frame data, and then output a target image frame.

[0065] In the device of the above-mentioned embodiment of the present invention, the frame data captured by the camera is processed, and non-continuous frame data are discarded to ensure the continuity of the frame data of the spliced ​​image frame; the occurrence of screen distortion, blurring, etc. caused by the splicing of non-continuous frame data is avoided, the imaging clarity is improved, and the display effect of the target image frame is guaranteed.

[0066] For the specific definition of the image data processing device, please refer to the definition of the image data processing method above, which will not be repeated here. Each module in the above-mentioned image data processing device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0067] The present invention also provides an electronic device, comprising a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to execute the computer program to implement the above-mentioned image data processing method. Further, the internal structure diagram of the electronic device can be as follows: Figure 3 As shown, the electronic device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a readable storage medium and an internal memory. The readable storage medium stores an operating system, computer-readable instructions and a database. The internal memory provides an environment for the operation of the operating system and computer-readable instructions in the readable storage medium. When the computer-readable instructions are executed by the processor, an image data processing method is implemented. The readable storage medium provided in this embodiment includes a non-volatile readable storage medium and a volatile readable storage medium.

[0068] In one embodiment, the electronic device can be a SOC in a vehicle's on-board system for controlling a vehicle camera, or a SOC in a traffic system for managing traffic electronic eyes. The SOC can also be a SOC in an electronic product for controlling a camera, etc.; as long as the installation object of the SOC is equipped with a camera, and the SOC needs to be used to execute the image data processing method of the present invention to ensure the image display effect.

[0069] The present invention also provides a vehicle system, further, the vehicle system also includes a first interface arranged in the kernel layer of the SOC and used to connect the end of the deserializer away from the serializer, a second interface connected between the serializer and the deserializer, and a third interface connected between the odd-even camera and the serializer; it is understandable that the first interface. Further, the first interface, the second interface, and the third interface are all mobile industry processor serial interfaces. In one embodiment, the vehicle system also includes a first port connected between the odd-even camera and the serializer, a second port connected between the serializer and the deserializer, and a third port arranged in the kernel layer of the SOC and used to connect the end of the deserializer away from the serializer. The first port, the second port, and the third port are all general input and output ports. For the specific definition of the vehicle system, please refer to the definition of the image data processing method and SOC above, which will not be repeated here.

[0070] The present invention also provides a vehicle, including the above-mentioned electronic device. Further, the vehicle includes a camera, a serializer, and a deserializer. It is understandable that the vehicle also includes a first interface provided in the core layer of the electronic device and used to connect the end of the deserializer away from the serializer, a second interface connected between the serializer and the deserializer, and a third interface connected between the odd-even camera and the serializer; further, the first interface, the second interface, and the third interface are all mobile industry processor serial interfaces. In one embodiment, the vehicle also includes a first port connected between the odd-even camera and the serializer, a second port connected between the serializer and the deserializer, and a third port provided in the core layer of the electronic device and used to connect the end of the deserializer away from the serializer. Among them, the first port, the second port, and the third port are all general input and output ports. For the specific features of the vehicle, please refer to the above definition of the image data processing method and the electronic device, which will not be repeated here. In the vehicle of the above-mentioned embodiment of the present invention, the electronic device processes the frame data captured by the camera and drops the non-continuous frame data to ensure the continuity of the frame data of the spliced ​​image frame; it avoids the occurrence of screen distortion, blurring, etc. caused by the splicing of non-continuous frame data, improves the imaging clarity, and ensures the display effect of the target image frame.

[0071] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions implement the above-mentioned image data processing method when executed by a processor.

[0072] The present invention also provides a computer program product, including a computer program, wherein the computer program implements the above-mentioned image data processing method when executed by a processor. The specific features of the computer program product can be found in the above definition of the image data processing method, which will not be repeated here.

[0073] The present invention also provides a vehicle, including the above-mentioned computer program product. For the specific features of the computer program product of the vehicle, please refer to the definition of the image data processing method above, which will not be repeated here. In the vehicle of the above-mentioned embodiment of the present invention, when the computer program of the computer program product is executed by the processor, the frame data captured by the camera can be processed, and the non-continuous frame data can be dropped to ensure the continuity of the frame data of the spliced ​​image frame; the occurrence of screen distortion, blurring, etc. caused by the splicing of non-continuous frame data is avoided, the imaging clarity is improved, and the display effect of the target image frame is ensured.

[0074] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a readable storage medium, and the readable storage medium includes a non-volatile readable storage medium and a volatile readable storage medium. When the computer-readable instructions are executed, they may include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0075] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units or modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional units or modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0076] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for processing image data, It is characterized in that The image data processing method is applied to SOC, and the SOC is applied to the system; The image data processing method comprises: receiving frame data captured by a camera, wherein the frame data comprises image frame data and image frame data identifiers; the image frame data identifiers comprise odd frames and even frames; Marking the system receiving time as the time information of the frame data, and determining two frames of the image frame data with adjacent system receiving times as a comparison frame group; in two adjacent comparison frame groups, the image frame data with a later system receiving time in the former comparison frame group and the image frame data with an earlier system receiving time in the latter comparison frame group are the same image data frame; When it is determined that the image frame data identifiers corresponding to the two frames of the image frame data in the comparison frame group are not the same, and it is determined according to the time information that the difference of the time information corresponding to the two frames of the image frame data in the comparison frame group is less than or equal to the preset time threshold, confirming that the two frames of the image data in the comparison frame group are not discontinuous frame data; If it is determined according to the time information of the frame data and the image frame data identifier that there are non-continuous comparison frame groups in the two adjacent comparison frame groups, the image frame data with an earlier system reception time in the non-continuous comparison frame groups is deleted, and the target image frame is output according to the remaining comparison frame groups that are not non-continuous frame data; wherein the outputting of the target image frame includes: Filling the odd frame rows and even frame rows in the comparison frame group that are the remaining non-continuous frame data into the target buffer alternately row by row; wherein the odd frame row refers to each row of odd frame data in the frame data of the odd frame; and the even frame row refers to each row of even frame data in the frame data of the even frame; After confirming that all the odd frame rows and the even frame rows of the comparison frame group that are not non-continuous frame data are filled into the target buffer, all the odd frame rows and the even frame rows in the target buffer are merged to generate a target image frame corresponding to the comparison frame group that is not non-continuous frame data.

2. The image data processing method according to claim 1, It is characterized in that The non-continuous frame data includes: a difference in time information of two adjacent frame data is greater than a preset time threshold; and / or two adjacent frame data do not include an odd frame and an even frame.

3. The image data processing method according to claim 2, It is characterized in that The output target image frame includes: splicing the image frame data of the odd frame with the image frame data of the even frame at a previous moment and / or the image frame data of the even frame at a next moment to obtain and output the target image frame.

4. The image data processing method according to claim 3, It is characterized in that The image data processing method further includes: controlling the camera to restart when the target image frame is not output within a preset time interval.

5. The image data processing method according to any one of claims 1 to 4, It is characterized in that The receiving frame data captured by the camera includes: The image frame data is deserialized, the deserialized image frame data is received through a first preset interface, and the image frame data identifier is received through a second preset interface; the first preset interface and the second preset interface are different types of interfaces.

6. An electronic device comprising a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to execute the computer program to implement the image data processing method according to any one of claims 1 to 5.

7. A computer-readable storage medium storing computer-readable instructions. It is characterized in that When the computer-readable instructions are executed by a processor, the image data processing method according to any one of claims 1 to 5 is implemented.

8. A computer program product, It is characterized in that The invention comprises a computer program, which implements the image data processing method according to any one of claims 1 to 5 when being executed by a processor.

9. A vehicle, It is characterized in that Comprising the electronic device as claimed in claim 6.

Images