Image processing method and device, electronic equipment and storage medium

Abstract

The invention provides an image processing method and device, electronic equipment and a storage medium, and relates to the technical field of computer processing.The method comprises the steps that multiple pieces of first image data are obtained, each piece of first image data comprises a data head and effective image data, and for each piece of first image data in at least part of the first image data, the effective image data are obtained; the data heads of the first image data are located at the tails of the first image data, the multiple pieces of first image data are synthesized into an image through the first preset rule, and due to the fact that the data heads of at least part of the first image data are located at the tails of the first image data, when the multiple pieces of first image data are spliced, the splicing efficiency is improved. The data head of the previous image data can be covered by the effective image data of the next image data, so that the time delay caused by firstly extracting the data head and then splicing each time can be reduced, and the data transmission efficiency of the picture on the USB bus can be improved.

Description

Technical Field

[0001] The present invention relates to the field of computer processing technology, and in particular to an image processing method, device, electronic equipment and storage medium. Background Art

[0002] The USB video class (UVC) is a protocol standard defined for universal serial bus (USB) video capture devices. USB cameras are also implemented according to this protocol standard. For some original images, such as luminance-color difference (YUV) original images, the amount of data in the image is larger than the amount of data that can be transmitted on the USB bus in a single time. Therefore, it is necessary to transmit the data on the USB bus multiple times to complete the data transmission of the original image.

[0003] However, the current method of transmitting data multiple times on the USB bus has the problem of low data transmission efficiency of images. Summary of the invention

[0004] In view of this, an object of the present invention is to provide an image processing method, device, electronic device and storage medium, which can improve the data transmission efficiency of pictures on a USB bus.

[0005] In a first aspect, an embodiment of the present application provides an image processing method, comprising: acquiring multiple first image data acquired by an image acquisition device, the multiple first image data corresponding to the same image, each first image data comprising a data header and valid image data, for each first image data in at least part of the first image data, the data header of the first image data is located at the end of the first image data, the data header is used to define information of the first image data, and the valid image data is used to synthesize an image; synthesizing the multiple first image data into an image according to a first preset rule.

[0006] In one possible implementation, at least part of the first image data includes at least two first image data among multiple first image data arranged according to receiving time, and the multiple first image data are synthesized into an image according to a first preset rule, including: for any two first image data among the at least two first image data, the valid image data of the first target image data among the any two first image data is overwritten with the data header of the second target image data among any two first image data, so that the valid image data of the first target image data and the valid image data of the second target image data are spliced ​​into spliced ​​valid image data, and the receiving time of the first target image data is later than the receiving time of the second target image data; and the image is synthesized based on the spliced ​​valid image data.

[0007] In a possible implementation, before synthesizing multiple first image data into an image through a first preset rule, it also includes: extracting a data header of at least one first image data; verifying the multiple first image data based on the data header of at least one first image data; synthesizing the multiple first image data into an image through the first preset rule, including: in response to the result that the multiple first image data passed the verification, synthesizing the valid image data in the multiple first image data into an image through the first preset rule.

[0008] In one possible implementation, before synthesizing multiple first image data into an image through a first preset rule, it also includes: querying the image data processing protocol supported by the image acquisition device; synthesizing the multiple first image data into an image through the first preset rule, including: in response to querying that the image data processing protocol supported by the image acquisition device includes the first image data processing protocol, synthesizing the multiple first image data into an image through the first preset rule.

[0009] In one possible implementation, a data header of a first first image data among multiple first image data includes flag information, and the flag information is used to indicate an image data processing protocol supported by an image acquisition device, a reception time of the first first image data is earlier than a reception time of other first image data, and the other first image data include data other than the first first image data among the multiple first image data; querying the image data processing protocol supported by the image acquisition device includes: in response to the flag information being the first flag, determining that the image data processing protocol supported by the image acquisition device includes the first image data processing protocol.

[0010] In a possible implementation, the method is applied to a processor, the processor includes firmware, a shared memory, and an application program, and multiple first image data are synthesized into an image according to a first preset rule, including:

[0011] The firmware synthesizes a plurality of first image data into an image according to a first preset rule; the method further includes: the firmware writes the image into a shared memory; the application obtains the image from the shared memory and displays the image.

[0012] In a possible implementation manner, the firmware writes the image into the shared memory, including: the firmware writes the image into the shared memory through direct memory access DMA.

[0013] In second aspect, an embodiment of the present application provides an image processing method, including: acquiring data of the same image; sending multiple first image data to a processor, the multiple first image data corresponding to an image, each first image data including a data header and valid image data, for each first image data in at least part of the first image data, the data header of the first image data is located at the end of the first image data, the data header is used to define information of the first image data, and the valid image data is used to synthesize the image.

[0014] In one possible implementation, before sending multiple first image data to the processor, it also includes: querying the image data processing protocol supported by the processor; sending multiple first image data to the processor, including: in response to querying that the image data processing protocol supported by the processor includes the first image data processing protocol, sending multiple first image data to the processor.

[0015] In one possible implementation, the method also includes: in response to a query that the image data processing protocol supported by the processor does not include the first image data processing protocol, sending multiple second image data to the processor, the multiple second image data corresponding to an image, each second image data including a data header and valid image data, and the data header in the second image data is located at the head of the second image data.

[0016] In a third aspect, an embodiment of the present application provides an image processing device, comprising: an acquisition module, used to acquire multiple first image data acquired by an image acquisition device, the multiple first image data correspond to the same image, each first image data includes a data header and valid image data, and for each first image data in at least part of the first image data, the data header of the first image data is located at the end of the first image data, the data header is used to define information of the first image data, and the valid image data is used to synthesize an image; a processing module, used to synthesize the multiple first image data into an image according to a first preset rule.

[0017] In a fourth aspect, an embodiment of the present application provides an image processing device, comprising: an acquisition module, used to acquire data of the same image; a sending module, used to send multiple first image data to a processor, the multiple first image data corresponding to an image, each first image data including a data header and valid image data, for each first image data in at least part of the first image data, the data header of the first image data is located at the end of the first image data, the data header is used to define information of the first image data, and the valid image data is used to synthesize the image.

[0018] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the method in the first aspect or the second aspect and any possible implementation thereof.

[0019] In the sixth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the method in the first aspect or the second aspect and any possible implementation method thereof.

[0020] The embodiments of the present invention bring the following beneficial effects:

[0021] By acquiring multiple first image data acquired by an image acquisition device, the multiple first image data correspond to the same image, each first image data includes a data header and valid image data, and for each first image data in at least part of the first image data, the data header of the first image data is located at the end of the first image data, the data header is used to define the information of the first image data, and the valid image data is used to synthesize the image; the multiple first image data are synthesized into an image according to a first preset rule, and since the data header of at least part of the first image data is located at the end of the first image data, when splicing the multiple first image data, it is beneficial to cover the data header of the previous image data with the valid image data of the next image data, so that the delay caused by extracting the data header and then splicing each time can be reduced, thereby helping to improve the data transmission efficiency of the picture on the USB bus. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 A flowchart of an image processing method provided in an embodiment of the present application;

[0024] Figure 2 A schematic diagram of splicing effective image data provided by an embodiment of the present application;

[0025] Figure 3 A schematic diagram of a processor framework provided in an embodiment of the present application;

[0026] Figure 4 A flowchart of another image processing method provided in an embodiment of the present application;

[0027] Figure 5 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0028] Figure 6 A schematic diagram of the structure of another image processing device provided in an embodiment of the present application;

[0029] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0031] At present, for some original pictures, such as YUV original pictures, the amount of data in the picture is larger than the amount of data that can be transmitted on the USB bus in a single time. Therefore, it is necessary to transmit the data multiple times on the USB bus to complete the data transmission of the original picture. For example: 1920*1200 resolution, where the load size is 1920*1200*2=4608000, a single transmission on the USB bus is usually much smaller than this amount, such as 16KB. The UVC protocol stipulates that the first 12 bytes of each transmission are the UVC protocol header. After receiving the data, the USB host mode (Host) end needs to remove the 12-byte protocol header and then assemble it into a complete photo. Generally speaking, in the vehicle-mounted system, this part of the assembly work is completed by the central processing unit (CPU), which brings two additional overheads: 1. CPU resource overhead. 2. Latency overhead.

[0032] When there are fewer cameras, users may not feel it. In the autonomous driving system, more than a dozen cameras are deployed. If this part of the cost can be saved, it will bring considerable benefits.

[0033] In view of this, the embodiments of the present application provide an image processing method, device, electronic device and storage medium, which can improve the data transmission efficiency of pictures on a USB bus.

[0034] In general, the embodiment of the present application changes the position of the 12-byte UVC header of the standard protocol and utilizes the direct memory access (DMA) feature of the USB bus controller to automatically verify the USB camera data during the DMA process and automatically copy the UVC data header (also known as the protocol header), thereby completing automatic assembly, saving CPU resources and reducing latency.

[0035] The image processing method provided in the embodiment of the present application can be applied to a processor, an image acquisition device, or an image processing system including an image acquisition device and a processor.

[0036] See also Figure 1 , Figure 1 A flowchart of an image processing method provided in an embodiment of the present application. Figure 1 The method shown can be applied to a processor, and the image processing method may include:

[0037] S110. Acquire multiple first image data acquired by an image acquisition device, where the multiple first image data correspond to the same image, and each first image data includes a data header and valid image data. For each first image data in at least part of the first image data, the data header of the first image data is located at the end of the first image data.

[0038] In this embodiment, the plurality of first image data can be understood as a plurality of image data corresponding to one frame of image. In the first image data, the data header of the first image data is located at the end of the first image data, that is, the valid image data (payload) in the first image data is located in front of the data header. The image in this embodiment can be an independent image or one of the frames of image in a video, which is not limited here.

[0039] It should be noted that the front and back in this embodiment can be the order of parsing when parsing data. For example, the part parsed first is in front of the part parsed later, in other words, the part parsed later is behind the part parsed first.

[0040] Among them, the image acquisition device can be a device that acquires an image to obtain image data. In this embodiment, the amount of data that the image acquisition device can transmit at a single time is less than the complete data amount of a frame of image. Exemplarily, the image acquisition device can be, for example, a USB camera, or a camera of other data protocols, which is not limited here. The data header is used to define the information of the first image data, for example, it can define the pixel points of the image and the data amount of the image data corresponding to the image. The effective image data is used to synthesize the image, for example, the effective image data can indicate the information of the pixel points of the image.

[0041] S120 , synthesizing a plurality of first image data into an image according to a first preset rule.

[0042] In the embodiment of the present application, after receiving a plurality of first image data, an image may be synthesized using the plurality of first data.

[0043] In this embodiment, multiple first image data acquired by an image acquisition device are acquired, the multiple first image data correspond to the same image, each first image data includes a data header and valid image data, and for each first image data in at least part of the first image data, the data header of the first image data is located at the end of the first image data, the data header is used to define information of the first image data, and the valid image data is used to synthesize the image; the multiple first image data are synthesized into an image according to a first preset rule, and since the data header of at least part of the first image data is located at the end of the first image data, when splicing the multiple first image data, it is beneficial to cover the data header of the previous image data with the valid image data of the next image data, so that the delay caused by extracting the data header and then splicing each time can be reduced, thereby helping to improve the data transmission efficiency of the picture on the USB bus.

[0044] In a possible implementation manner, at least part of the first image data includes at least two first image data arranged according to the receiving time among the plurality of first image data, and the plurality of first image data are synthesized into an image according to a first preset rule, including:

[0045] For any two of the at least two first image data, the valid image data of the first target image data in any two first image data overwrites the data header of the second target image data in any two first image data, so that the valid image data of the first target image data and the valid image data of the second target image data are spliced ​​into spliced ​​valid image data, and the receiving time of the first target image data is later than the receiving time of the second target image data; then, an image is synthesized based on the spliced ​​valid image data.

[0046] In this embodiment, if the receiving time of the first target image data is later than the receiving time of the second target image data, the processor can overwrite the data header of the previously received image data with the valid image data of the later received image data, so that the valid data of the two adjacent received image data can be spliced ​​together.

[0047] See also Figure 2 , Figure 2 A schematic diagram of splicing effective image data provided in an embodiment of the present application.

[0048] like Figure 2 As shown, the first target image data and the second target image data are two adjacent image data, and then the effective image data of the second target image data overwrites the data header of the first target image data, thereby obtaining the spliced ​​effective image data.

[0049] In this embodiment, the effective image data of the first target image data in any two first image data is overwritten with the data header of the second target image data in any two first image data, so that the effective image data of the first target image data and the effective image data of the second target image data are spliced ​​into spliced ​​effective image data, and the receiving time of the first target image data is later than the receiving time of the second target image data; an image is synthesized based on the spliced ​​effective image data, and since the spliced ​​effective image data is obtained by using two adjacent image data, the efficiency of data processing can be improved.

[0050] It should be noted that the valid image data of the first image data other than the first first image data among the multiple first image data are all located in front of the data header. The data header of the first first image data can be located at the head or tail of the first first image data, which is not limited here.

[0051] In another possible implementation manner, at least part of the non-adjacent image data of the first image data may be used for splicing.

[0052] In a possible implementation manner, before synthesizing the plurality of first image data into an image according to the first preset rule, the method further includes:

[0053] Extracting a data header of at least one first image data; and verifying a plurality of first image data based on the data header of at least one first image data.

[0054] Accordingly, synthesizing the plurality of first image data into an image according to the first preset rule includes:

[0055] In response to the result that the plurality of first image data pass the verification, valid image data in the plurality of first image data are synthesized into an image according to a first preset rule.

[0056] In this embodiment, by extracting the data header of at least one first image data; verifying multiple first image data based on the data header of at least one first image data, and in response to the results of multiple first image data passing the verification, the valid image data in the multiple first image data are synthesized into an image according to a first preset rule. In this way, processing can be performed after the image data passes the verification, which can reduce the situation of invalid processing caused by directly processing the image, which is beneficial to improving the effectiveness of image processing and improving the effective utilization of system resources.

[0057] It should be noted that at least the data header of the first image data may include the data header of the first first image data. Optionally, the data header of other first image data may also be included, which is not limited here. Optionally, one possible verification method may be, for example, whether the number of first image data indicated by the data header of the first first image data is consistent with the number of first image data actually received. If they are consistent, the verification is passed.

[0058] It should be understood that other verification methods are possible and can be set as needed, and are not limited here.

[0059] In another possible implementation, image processing may be performed directly without verification, thus improving the efficiency of image processing.

[0060] Before synthesizing the plurality of first image data into an image according to the first preset rule, the method further includes:

[0061] Query the image data processing protocols supported by the image acquisition device.

[0062] Accordingly, synthesizing the plurality of first image data into an image according to the first preset rule includes:

[0063] In response to finding that the image data processing protocols supported by the image acquisition device include a first image data processing protocol, a plurality of first image data are synthesized into an image according to a first preset rule.

[0064] The first image data processing protocol may refer to a self-developed protocol, for example, it may be a data protocol format in which the data header is placed at the end of the first image data in this embodiment.

[0065] In this embodiment, if the image acquisition device can transmit the first image data processing protocol by default based on supporting the first image data processing protocol, the processor can process multiple first image data to synthesize an image, which can improve the efficiency of data processing.

[0066] In another possible implementation, when the processor finds that the image data processing protocols supported by the image acquisition device include the first image data processing protocol, the processor may inform the image processing device that the processor also supports the first image data processing protocol. Thus, when both the processor and the image acquisition device support the first image data processing protocol, image processing is performed using the technical solution of the embodiment of the present application, which can improve the success rate of image processing.

[0067] In one possible implementation, a data header of a first first image data among multiple first image data includes flag information, and the flag information is used to indicate an image data processing protocol supported by an image acquisition device, and a receiving time of the first first image data is earlier than a receiving time of other first image data, and the other first image data includes data other than the first first image data among the multiple first image data.

[0068] Accordingly, query the image data processing protocols supported by the image acquisition device, including:

[0069] In response to the flag bit information being the first flag bit, it is determined that the image data processing protocols supported by the image acquisition device include the first image data processing protocol.

[0070] In this embodiment, by carrying flag information indicating the image data processing protocol supported by the image acquisition device in the first first image data header, when the first first image data is received, it can be known whether the image data processing protocol supported by the image acquisition device includes the first image data processing protocol. In this way, image processing can be performed in a timely manner through the solution of the embodiment of the present application without the need for additional query signaling, and the signaling overhead required for image processing can also be reduced.

[0071] In another possible implementation, the processor may query the image data processing protocol supported by the image acquisition device through an additional signaling, and then the image acquisition device informs the processor of the image data processing protocol supported by it through an additional response signaling, thereby reducing the transmission resources required to transmit the first image data.

[0072] In a possible implementation, the processor includes firmware, a shared memory, and an application program, and synthesizes a plurality of first image data into an image according to a first preset rule, including:

[0073] The firmware synthesizes a plurality of first image data into an image according to a first preset rule.

[0074] Correspondingly, the image processing method further includes: the firmware writes the image into the shared memory; the application obtains the image from the shared memory and displays the image.

[0075] In this embodiment, after the processor obtains multiple first image data, the firmware synthesizes the multiple first image data into an image according to a first preset rule, and then writes the image into a shared memory through the firmware, and the application obtains the image from the shared memory and displays the image.

[0076] In a possible implementation, the firmware writes the image to the shared memory, including:

[0077] The firmware writes the image to the shared memory via direct memory access (DMA).

[0078] In this embodiment, the image is written into the shared memory through direct memory access (DMA) by the firmware, so that the efficiency of image processing can be improved, and further the efficiency of image display can be improved.

[0079] See also Figure 3 , Figure 3 A schematic diagram of a processor framework provided in an embodiment of the present application. Figure 3 The processor shown may include a user layer (user) and a kernel layer (kernel).

[0080] User camera application: can process buffers and display (process buffer & display): The user camera application is responsible for processing the video frames captured from the camera and displaying them on the screen. The application takes completed video frame buffers from a queue, which is managed by the videobuf2 component in video for Linux 2, and displays these frames. V4L2 videobuf2 provides buffer management functions. buffer management This is a video buffer management framework used to manage the buffers of video capture devices in the Linux kernel. It provides a queue mechanism that allows applications to take completed buffers from the queue and put new (or used) buffers back to the queue. Specifically, the application can complete queue operations (queue operations) and take out completed buffers (dequeue completedbuffer). Queue operations: The application takes the completed captured video frame buffers from the queue managed by V4L2 videobuf2. Enqueue used buffer (put in used buffer): The application puts the displayed (or processed) video frame buffers back to the queue so that V4L2 videobuf2 can reuse them.

[0081] Shared memory: The application maps the video frames into memory through a shared memory mechanism, which allows data to be shared efficiently between different processes or threads. This usually involves memory mapping between kernel space and user space.

[0082] Zero copy: Zero copy technology is a method to reduce the number of times data is copied between memories, which can improve the efficiency of data transmission. Figure 3 In , zero-copy technology is used in the process of capturing video frames from the camera to displaying them to reduce unnecessary memory copies.

[0083] Define and implement application programming interfaces (APIs), where APIs can include define APIs and implement APIs. Specifically, for define APIs: a set of APIs needs to be defined between the application and the kernel so that they can communicate and collaborate with each other. For implement APIs, these APIs need to be implemented in the application and the kernel to ensure that they can work correctly.

[0084] Async Transfer by CPU Threads: Asynchronous transfer technology allows CPU threads to process data in the background without blocking the main thread. Figure 3 In the embodiment, the asynchronous transmission may be a process of capturing a video frame from a camera and placing it into a queue. In this embodiment, the asynchronous transmission may be a transmission mode of a standard mode, such as a mode of performing data processing using a second image data processing protocol, which is not limited here. The second image data processing protocol may be, for example, a standard protocol.

[0085] In summary, this diagram shows the process of how a user camera application processes video frames, including steps such as capturing video frames from the camera, managing buffers, mapping video frames through shared memory, defining and implementing APIs, transferring data asynchronously, communicating with a UVC camera, transferring data through URB and DMA, and handling completion callbacks.

[0086] In general, in an embodiment of the present application, the position of the UVC header and the payload during USB transmission on the camera side is modified from the head to the tail. In addition, the camera-side Firmware is modified to provide a USB query command, and the host can ask the USB camera transmission end whether it is working in the standard UVC protocol mode (the second image data processing protocol) or the first image data processing protocol. In addition, the host-side UVC kernel driver module is also modified to automatically adjust the memory position of the payload when sending a USB bus transmission request. In addition, the host-side USB bus controller kernel driver module is modified so that its serial DMA address just automatically overwrites the UVC header of the previous payload. In addition, the upper-level camera application software is modified to automatically use the interface to query the transmission mode of the camera side. If it is in the standard UVC protocol mode, it enters the standard mode. If the camera side works in the self-developed solution mode, it switches to the processing process related to the first image data processing protocol to process the image data.

[0087] See also Figure 4 , Figure 4 A flowchart of another image processing method provided by an embodiment of the present application. The method of this embodiment can be applied to an image acquisition device, and the method may include:

[0088] S410: Collect data of the same image.

[0089] S420. Send multiple first image data to the processor, the multiple first image data correspond to images, each first image data includes a data header and valid image data, and for each first image data in at least part of the first image data, the data header of the first image data is located at the end of the first image data, the data header is used to define information of the first image data, and the valid image data is used to synthesize the image.

[0090] This embodiment can refer to the relevant description of the above embodiment, which will not be described in detail here.

[0091] In this embodiment, multiple first image data acquired by an image acquisition device are acquired, the multiple first image data correspond to the same image, each first image data includes a data header and valid image data, and for each first image data in at least part of the first image data, the data header of the first image data is located at the end of the first image data, the data header is used to define information of the first image data, and the valid image data is used to synthesize the image; the multiple first image data are synthesized into an image according to a first preset rule, and since the data header of at least part of the first image data is located at the end of the first image data, when splicing the multiple first image data, it is beneficial to cover the data header of the previous image data with the valid image data of the next image data, so that the delay caused by extracting the data header and then splicing each time can be reduced, thereby helping to improve the data transmission efficiency of the picture on the USB bus.

[0092] In a possible implementation manner, before sending the plurality of first image data to the processor, the method further includes:

[0093] Query the image data processing protocols supported by the processor.

[0094] Accordingly, a plurality of first image data are sent to the processor, including:

[0095] In response to finding that the image data processing protocols supported by the processor include the first image data processing protocol, a plurality of first image data are sent to the processor.

[0096] In this embodiment, when it is found that the image data processing protocol supported by the processor includes the first image data processing protocol, multiple first image data are sent to the processor, which can improve the success rate of image processing and reduce image processing anomalies caused by data processing protocol mismatch.

[0097] In another possible implementation, the image processing method further includes:

[0098] In response to querying that the image data processing protocol supported by the processor does not include the first image data processing protocol, multiple second image data are sent to the processor, the multiple second image data correspond to images, each second image data includes a data header and valid image data, and the data header in the second image data is located at the head of the second image data.

[0099] In this embodiment, if it is found that the image data processing protocol supported by the processor does not include the first image data processing protocol, it means that the processor supports the default second image data processing protocol. Therefore, multiple second image data are sent to the processor, and each second image data includes a data header and valid image data. The data header in the second image data is located at the head of the second image data. In this way, the processor can extract the data header from the second image data and then extract the valid image data for splicing.

[0100] Therefore, in this embodiment, the CPU resources for UVC header parsing and image data assembly can be saved. In addition, the total delay of USB camera transmission is reduced. In addition, the method automatically identifies whether the camera is in UVC standard protocol mode or self-developed solution mode, so that it can automatically be compatible with cameras in both modes (also known as cameras or image acquisition devices).

[0101] See also Figure 5 , Figure 5 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application is shown in FIG. Figure 5 The device may be applied to a processor, and the device may include:

[0102] An acquisition module 510 is used to acquire a plurality of first image data acquired by an image acquisition device, wherein the plurality of first image data correspond to the same image, and each first image data includes a data header and valid image data. For each first image data in at least part of the first image data, the data header of the first image data is located at the end of the first image data, and the data header is used to define information of the first image data, and the valid image data is used to synthesize an image;

[0103] The processing module 520 is used to synthesize a plurality of first image data into an image according to a first preset rule.

[0104] The device provided in the embodiment of the present invention has the same technical features as the method executed by the processor provided in the above embodiment, so it can also solve the same technical problems and achieve the same technical effects, which will not be described in detail here.

[0105] See also Figure 6 , Figure 6 A structural diagram of another image processing device provided in an embodiment of the present application is shown in FIG. Figure 6 The device can be applied to an image acquisition device, and the device may include:

[0106] The acquisition module 610 is used to acquire data of the same image;

[0107] The sending module 620 is used to send multiple first image data to the processor, the multiple first image data correspond to images, each first image data includes a data header and valid image data, and for each first image data in at least part of the first image data, the data header of the first image data is located at the end of the first image data, the data header is used to define the information of the first image data, and the valid image data is used to synthesize the image.

[0108] The device provided in the embodiment of the present invention has the same technical features as the method performed by the image acquisition device provided in the above embodiment, so it can also solve the same technical problems and achieve the same technical effects, which will not be described in detail here.

[0109] This embodiment also provides an electronic device, including a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the above-mentioned image processing method. The electronic device can be a server, a terminal device, or an image acquisition device.

[0110] See also Figure 7 As shown, the electronic device includes a processor 100 and a memory 101. The memory 101 stores computer executable instructions that can be executed by the processor 100. The processor 100 executes the computer executable instructions to implement the steps of the above method.

[0111] Further, Figure 7 The electronic device shown further includes a bus 102 and a communication interface 103 , and the processor 100 , the communication interface 103 and the memory 101 are connected via the bus 102 .

[0112] The memory 101 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 103 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used. The bus 102 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0113] The processor 100 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 100. The above processor 100 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present invention can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 101, and the processor 100 reads the information in the memory 101 and completes the steps of the method of the above embodiment in combination with its hardware.

[0114] The processor in the above electronic device can implement the steps in any of the above image processing methods by executing computer executable instructions.

[0115] This embodiment also provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the steps in any of the above-mentioned image processing methods.

[0116] The computer-executable instructions stored in the computer-readable storage medium can implement the steps in any of the above-mentioned image processing methods by executing the computer-executable instructions.

[0117] This embodiment also provides a computer program product, including program code. The instructions included in the program code can be used to execute the method in the previous method embodiment. The specific implementation can be found in the method embodiment, which will not be repeated here.

[0118] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0119] In addition, in the description of the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0120] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0121] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0122] Finally, it should be noted that the above embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above embodiments, those skilled in the art should understand that any person skilled in the art can still modify the technical solutions recorded in the above embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. An image processing method, characterized in that: include: Acquire a plurality of first image data acquired by an image acquisition device, wherein the plurality of first image data correspond to the same image, each of the first image data comprises a data header and valid image data, and for each first image data in at least part of the first image data, the data header of the first image data is located at the end of the first image data, the data header is used to define information of the first image data, and the valid image data is used to synthesize the image; The plurality of first image data are synthesized into the image according to a first preset rule.

2. The method according to claim 1, characterized in that The at least part of the first image data includes at least two first image data arranged according to the receiving time among the plurality of first image data, and the synthesizing the plurality of first image data into the image according to the first preset rule includes: For any two of the at least two first image data, overwriting the data header of the second target image data in the any two first image data with the valid image data of the first target image data in the any two first image data, so that the valid image data of the first target image data and the valid image data of the second target image data are spliced ​​into spliced ​​valid image data, and the receiving time of the first target image data is later than the receiving time of the second target image data; The image is synthesized based on the spliced ​​valid image data.

3. The method according to claim 1, characterized in that Before synthesizing the plurality of first image data into the image according to the first preset rule, the method further includes: extracting at least one data header of the first image data; verifying the plurality of first image data based on the data header of at least one of the first image data; The step of synthesizing the plurality of first image data into the image according to a first preset rule comprises: In response to the result that the plurality of first image data pass the verification, the valid image data in the plurality of first image data are synthesized into the image according to a first preset rule.

4. The method according to claim 1, characterized in that Before synthesizing the plurality of first image data into the image according to the first preset rule, the method further includes: Querying the image data processing protocol supported by the image acquisition device; The step of synthesizing the plurality of first image data into the image according to a first preset rule comprises: In response to finding that the image data processing protocols supported by the image acquisition device include a first image data processing protocol, the plurality of first image data are synthesized into the image according to a first preset rule.

5. The method according to claim 4, characterized in that The data header of the first first image data among the plurality of first image data includes flag information, the flag information is used to indicate the image data processing protocol supported by the image acquisition device, the reception time of the first first image data is earlier than the reception time of other first image data, and the other first image data includes data other than the first first image data among the plurality of first image data; The querying of the image data processing protocol supported by the image acquisition device includes: In response to the flag information being the first flag, it is determined that the image data processing protocols supported by the image acquisition device include the first image data processing protocol.

6. The method according to any one of claims 1 to 5, characterized in that The method is applied to a processor, the processor includes firmware, a shared memory and an application program, and the step of synthesizing the plurality of first image data into the image according to a first preset rule includes: The firmware synthesizes the plurality of first image data into the image according to a first preset rule; The method further comprises: The firmware writes the image into the shared memory; The application obtains the image from the shared memory and displays the image.

7. The method according to claim 6, characterized in that The firmware writes the image into the shared memory, including: The firmware writes the image into the shared memory through direct memory access (DMA).

8. An image processing method, characterized in that: include: Acquire data from the same image; Send a plurality of first image data to the processor, the plurality of first image data corresponding to the image, each of the first image data comprising a data header and valid image data, for each first image data in at least part of the first image data, the data header of the first image data is located at the end of the first image data, the data header is used to define information of the first image data, and the valid image data is used to synthesize the image.

9. The method according to claim 8, characterized in that Before sending the plurality of first image data to the processor, the method further includes: Querying the image data processing protocol supported by the processor; The sending the plurality of first image data to the processor comprises: In response to finding that the image data processing protocols supported by the processor include the first image data processing protocol, the plurality of first image data are sent to the processor.

10. The method according to claim 9, characterized in that The method further comprises: In response to querying that the image data processing protocol supported by the processor does not include the first image data processing protocol, multiple second image data are sent to the processor, the multiple second image data correspond to the image, each of the second image data includes the data header and the valid image data, and the data header in the second image data is located at the head of the second image data.

11. An image processing device, characterized in that: include: an acquisition module, configured to acquire a plurality of first image data acquired by an image acquisition device, wherein the plurality of first image data correspond to the same image, each of the first image data comprises a data header and valid image data, and for each first image data in at least part of the first image data, the data header of the first image data is located at the end of the first image data, the data header is used to define information of the first image data, and the valid image data is used to synthesize the image; The processing module is used to synthesize the plurality of first image data into the image according to a first preset rule.

12. An image processing device, characterized in that: include: An acquisition module, used for acquiring data of the same image; A sending module is used to send the multiple first image data to the processor, the multiple first image data correspond to the image, each of the first image data includes a data header and valid image data, for each first image data in at least part of the first image data, the data header of the first image data is located at the end of the first image data, the data header is used to define the information of the first image data, and the valid image data is used to synthesize the image.

13. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the method of any one of claims 1 to 7, or implements the method of any one of claims 8 to 10.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the method of any one of claims 1-7, or implement the method of any one of claims 8-10.

Images