Image display method and system and storage medium

Abstract

The invention discloses an image display method and system and a storage medium, and the method comprises the steps: obtaining a first image set and a second image set, and storing the processed first image information and second image information in a cache region; the first image set is obtained by sampling at a first frequency, the second image set is obtained by sampling at a second frequency, and the first frequency is greater than the second frequency; the cache region is scanned at a third frequency, the first image and / or the second image are / is displayed, and the third frequency is larger than or equal to the first frequency. According to the method, the cache area is scanned at the third frequency larger than or equal to the highest sampling frequency (namely the first frequency), it is ensured that even if the sampling frequencies of the image set are different, the display content can be refreshed at the enough frequency, and therefore the problem of image jamming or display delay is avoided, and meanwhile high fluency and real-time performance of the image are ensured.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image display method, system and storage medium. Background Art

[0002] In video processing and display technology, sampling frequency is a crucial parameter that determines the quality and smoothness of the image. The sampling frequency directly affects the accuracy of capturing and representing information in the image. When two video sources with different sampling frequencies need to be combined into one image for display, a series of technical challenges will be encountered.

[0003] Currently, the method of timestamping each frame is usually used to synchronize two or more video sources with different sampling frequencies. The introduction of timestamps can ensure that the image frames of each area are processed and displayed at the correct time point, thereby avoiding image confusion or overlap. However, even with the use of timestamp synchronization technology, when two or more channels of images with different sampling frequencies are processed and displayed on the same screen, the frame rates of the two video sources may be inconsistent due to the difference in sampling frequencies. When merging and displaying, if a frame fails to render in time, it will cause the picture to freeze, affecting the user experience. Summary of the invention

[0004] One of the purposes of the present invention is to provide an image display method to solve the technical problem in the prior art that when images with different sampling frequencies are displayed on one image, freezes and display is not smooth due to the use of timestamps for frame synchronization display.

[0005] One of the objectives of the present invention is to provide an image display system.

[0006] One of the objects of the present invention is to provide a computer storage medium.

[0007] In order to achieve one of the above-mentioned objects of the invention, the present invention provides an image display method, including: obtaining a first image set and a second image set, and storing the processed first image information and the second image information in a cache area respectively; the first image set is sampled at a first frequency, and the second image set is sampled at a second frequency, and the first frequency is greater than the second frequency; scanning the cache area at a third frequency to display the first image and / or the second image, and the third frequency is greater than or equal to the first frequency.

[0008] As a further improvement of an embodiment of the present invention, a first image set and a second image set are obtained, and the processed first image set and second image set are respectively stored in a cache area, including: sampling at a first frequency to obtain the first image set, processing the first image set with a first operation, and storing a number of first image information at the first frequency in the first cache area according to the order in which the images are arranged in the set; at the same time, sampling at a second frequency to obtain the second image set, processing the second image set with a second operation, and storing a number of second image information at the second frequency in the second cache area according to the order in which the images are arranged in the set.

[0009] As a further improvement of an embodiment of the present invention, the method also includes: using the first thread synchronization lock to control the synchronous triggering of the sampling process of the first image set and the first operation processing process; using the second thread synchronization lock to control the synchronous triggering of the sampling process of the second image set and the second operation processing process.

[0010] As a further improvement of an embodiment of the present invention, the method includes: storing N first images processed by a first operation into a first shared buffer, and storing N first index values ​​corresponding to the N first images into a first cache area at a first frequency; the first index value is used to index the corresponding first image for scanning and displaying the first image; storing M second images processed by a second operation into a second shared buffer, and storing M second index values ​​corresponding to the M second images into a second cache area at a second frequency; the second index value is used to index the corresponding second image for scanning and displaying the second image.

[0011] As a further improvement of an embodiment of the present invention, the method includes: storing the sampled first image set in a third shared buffer, and generating N third index values ​​corresponding to the N first images at a first frequency; the third index value is used to index to the corresponding first image for implementing the first operation; storing the sampled second image set in a fourth shared buffer, and generating M fourth index values ​​corresponding to the M second images at a second frequency; the fourth index value is used to index to the corresponding second image for implementing the second operation.

[0012] As a further improvement of an embodiment of the present invention, scanning the cache area at a third frequency to display the first image and / or the second image includes: receiving a call instruction corresponding to P display modules, and scanning the cache area P times at a third frequency in response to the call instruction to display the current image until the next image is scanned and the display screen of the display module is updated with the next image, P≥1; when all P display modules display the next image, releasing the cache space in the cache area corresponding to the currently displayed image.

[0013] As a further improvement of an embodiment of the present invention, when P display modules all display the next image, the cache space corresponding to the currently displayed image in the cache area is released, including: initializing a reference count mark q corresponding to the current image, the reference count mark q is used to identify the number of times the current image is referenced; when a call instruction for the current image is received from the display module, the reference count mark q is increased by 1; when the display screen of the display module is updated to the next image, the reference count mark q is decreased by 1; and according to the reference count mark q, the cache space corresponding to the currently displayed image in the cache area is released.

[0014] As a further improvement of an embodiment of the present invention, the cache space corresponding to the currently displayed image in the cache area is released according to the reference count mark q, including: when the count mark q is reduced to 0, determining that P display modules all display the next image, and releasing the cache space corresponding to the currently displayed image in the cache area; wherein 0≤q≤P.

[0015] As a further improvement of one embodiment of the present invention, the method satisfies at least one of the following: the first image set contains information of the current image; and / or, the second image set contains information of the current image; the first image set contains information of the next image; and / or, the second image information contains information of the next image.

[0016] To achieve one of the above-mentioned purposes of the invention, one embodiment of the present invention provides an image display system, comprising: a first processing module, used to sample at a first frequency to obtain a first image set, and store the processed first image information in a cache area; a second processing module, used to sample at a second frequency to obtain a second image set, and store the processed second image information in a cache area; a display module, used to scan the cache area at a third frequency, and display the first image and / or the second image; a cache, used to establish the cache area; the first frequency is greater than the second frequency, and the third frequency is greater than or equal to the first frequency.

[0017] As a further improvement of an embodiment of the present invention, the first processing module includes a first sampling unit, a third shared buffer unit, a first processing unit, and a first shared buffer unit; the first sampling unit is used to sample at a first frequency to obtain a first image set; the third shared buffer unit is used to establish a third shared buffer, and the third shared buffer is used to store a third index value indexed to the first image to be processed; the first processing unit is used to perform a first operation; the first shared buffer unit is used to establish a first shared buffer, and the first shared buffer is used to store a first index value indexed to the processed first image. The second processing module includes a second sampling unit, a fourth shared buffer unit, a second processing unit, and a second shared buffer unit; the second sampling unit is used to sample at a second frequency to obtain a second image set; the fourth shared buffer unit is used to establish a fourth shared buffer, and the fourth shared buffer is used to store a fourth index value indexed to the second image to be processed; the second processing unit is used to perform a second operation, the second shared buffer unit is used to establish a second shared buffer, and the second shared buffer is used to store a second index value indexed to the processed second image.

[0018] To achieve one of the above-mentioned purposes of the invention, one embodiment of the present invention provides a computer storage medium, comprising: at least one processor; a memory, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, it executes the steps in the image display method.

[0019] Compared with the prior art, the embodiments of the present invention have at least one of the following beneficial effects:

[0020] The present invention adopts an image display method, such that a first image set and a second image set are obtained by acquiring at a first frequency and a second frequency, and the processed first image information and second image information are respectively stored in a cache area, and the cache area is scanned at a third frequency greater than or equal to the highest sampling frequency (i.e., the first frequency). The first image and / or the second image can be displayed in real time on a display module, ensuring that even if the sampling frequencies of the image sets are different, the display content can be refreshed at a sufficient frequency, thereby avoiding the problem of image freeze or display delay, while ensuring the high smoothness and real-time performance of the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a schematic diagram of the structure of an image display system in one embodiment of the present invention.

[0022] Figure 2 It is a structural schematic diagram of an image display mode in one embodiment of the present invention.

[0023] Figure 3It is a schematic diagram of the steps of an image display method in one embodiment of the present invention.

[0024] Figure 4 1 is a schematic diagram of step S1 in one embodiment of the present invention.

[0025] Figure 5 1 is a schematic diagram of step S11 in a specific embodiment of an embodiment of the present invention.

[0026] Figure 6 1 is a schematic diagram of step S11 in a specific embodiment of an embodiment of the present invention.

[0027] FIG. 7( a ) is a schematic diagram of the principle flow of an image display method in one embodiment of the present invention.

[0028] FIG. 7( b ) is a schematic diagram of the processing flow of the image processing channel in one embodiment of the present invention.

[0029] FIG. 7( c ) is a schematic diagram of the processing flow of the image display channel in one embodiment of the present invention.

[0030] Figure 8 2 is a schematic diagram of step S3 in one embodiment of the present invention.

[0031] Fig. 9 2 is a schematic diagram of step S32 in a specific embodiment of an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The present invention will be described in detail below in conjunction with the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, methodological, or functional changes made by a person skilled in the art based on these embodiments are all within the scope of protection of the present invention.

[0033] It should be noted that the term "comprises" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the description of the specific embodiments of the present invention, the terms "first", "second", "third", "fourth", etc. are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0034] like Figure 1 As shown, an embodiment of the present invention provides an image display system 100 .

[0035] The image display system 100 is a system capable of processing and displaying at least two images.

[0036] In one embodiment, the image display system 100 displays images or video sources with different sampling frequencies into one image on a display module smoothly according to an image display method without any lag.

[0037] In a specific embodiment, the image display method can be implemented with reference to any technical solution provided below.

[0038] The image display system 100 includes a first processing module 11 for collecting and processing a first channel of images (ie, corresponding to a first image set) from a first camera or a first video source. For example, the first channel of image set is an endoscopic image collected by an endoscope.

[0039] In one embodiment, the first processing module 11 is used to obtain a first image set by sampling at a first frequency, and store the processed first image information in a cache area.

[0040] The image display system 100 includes a second module 12 for collecting and processing a second channel of images (ie, corresponding to a second image set) from a second camera or a second video source. For example, the second channel of image set is ultrasound images collected by ultrasound equipment.

[0041] In one embodiment, the second processing module 12 is used to obtain a second image set by sampling at a second frequency, and store the processed second image information in a cache area.

[0042] The image display system 100 includes a display module 13 for displaying a first channel of images and a second channel of images.

[0043] In one embodiment, the display module 13 is used to scan the cache area at a third frequency, and to display the first image, or to display the second image, or to display the first image and the second image.

[0044] In one embodiment, the first frequency is greater than the second frequency, and the third frequency is greater than or equal to the first frequency.

[0045] In one embodiment, the first frequency is less than the second frequency, and the third frequency is greater than or equal to the second frequency.

[0046] In other words, by controlling the third frequency (i.e., display frequency) to be large enough so that it is greater than or equal to the maximum output frequency of the image to be displayed, the first image and the second image can be displayed as quickly as possible. The first image and the second image can be stored in different caches or in different buffers of the same cache, and the image display channel can scan the cache or buffer at the third frequency to display the corresponding image.

[0047] like Figure 2As shown, when displaying the first image and the second image, the display module 13 can support multiple display module displays, for example, supporting three display modules, the first display module displays the first image, the second display module displays the second image, and the third display module displays a combined image of the first image and the second image.

[0048] Continue to refer to Figure 2 As shown, the display module 13 can support multiple display modes when displaying the first image and the second image, such as supporting multiple display modes such as picture-in-picture and picture-outside-picture.

[0049] Among them, the picture-in-picture display mode includes the first image being displayed in a certain area of ​​the second image (such as the lower left corner), or the second image being displayed in a certain area of ​​the first image; the picture-outside-picture display mode includes the first image and the second image being displayed horizontally in the display area, that is, the first image can be displayed on the left side of the second image, or the first image can also be displayed on the right side of the second image.

[0050] Similarly, the picture-outside-picture display mode may also include the first image and the second image being displayed vertically in the display area, that is, the first image may be displayed on the upper side of the second image, or the first image may be displayed on the lower side of the second image. The present invention does not impose any specific limitation on this.

[0051] The image display system 100 may further include a buffer 14 for establishing the buffer area. The buffer area may be different buffer units in the same buffer 14 or buffer units in different buffers, and there is no specific limitation on this.

[0052] The relationship between the modules and functions provided above in the present invention is not absolutely fixed. In some embodiments, the units can be combined and integrated so that one module can assume the functions of multiple modules; in some embodiments, some functions can also be separated and implemented by several additional modules.

[0053] For example, the first processing module 11 and the second processing module 12 may be integrated into one processing module, which carries all or part of the functions of the above modules.

[0054] It should be noted that the first processing module 11 and the second processing module 12 can be understood as two image processing channels, which process two paths of image data in parallel and independently, and display them in different areas of the same display module.

[0055] In a specific embodiment, the first processing module includes a first sampling unit, a third shared buffer unit, a first processing unit, and a first shared buffer unit; the first sampling unit is used to sample at a first frequency to obtain a first image set; the third shared buffer unit is used to establish a third shared buffer, and the third shared buffer is used to store a third index value indexed to the first image to be processed; the first processing unit is used to perform a first operation; the first shared buffer unit is used to establish a first shared buffer, and the first shared buffer is used to store a first index value indexed to the processed first image.

[0056] In this embodiment, the first sampling unit may be an image sampling component or a first image sampling thread; similarly, the first processing unit may be an image processing component or a first image processing thread. The first shared buffer unit and the third shared buffer unit may be different storage areas on the same memory (or storage medium) or storage areas on different memories, and they may be static storage areas on the storage medium or storage areas dynamically created using the first image sampling thread or the first image processing thread.

[0057] Similarly, in a specific embodiment, the second processing module includes a second sampling unit, a fourth shared buffer unit, a second processing unit, and a second shared buffer unit; the second sampling unit is used to sample at a second frequency to obtain a second image set; the fourth shared buffer unit is used to establish a fourth shared buffer, and the fourth shared buffer is used to store a fourth index value indexed to the second image to be processed; the second processing unit is used to perform a second operation, the second shared buffer unit is used to establish a second shared buffer, and the second shared buffer is used to store a second index value indexed to the processed second image.

[0058] In this embodiment, the second sampling unit can be an image sampling component or a second image sampling thread; similarly, the second processing unit can be an image processing component or a second image processing thread. The second shared buffer unit and the fourth shared buffer unit can be different storage areas on the same memory (or storage medium) or storage areas on different memories. They can be static storage areas on the memory or storage areas dynamically created by the second image sampling thread or the second image processing thread.

[0059] like Figure 3 As shown, an embodiment of the present invention provides an image display method.

[0060] The image display method is applied in an image display system.

[0061] In one embodiment, the image display system may be as follows Figure 1As described above, the corresponding technical solution is set and referenced in the execution method provided by the present invention. Of course, the image display system used by the image display method provided by the present invention is not limited to this configuration structure.

[0062] like Figure 3 As shown, an image display method provided in one embodiment of the present invention includes the following steps.

[0063] Step S1, obtaining a first image set and a second image set, and storing the processed first image information and the second image information in a cache area respectively; the first image set is sampled at a first frequency, and the second image set is sampled at a second frequency, and the first frequency is greater than the second frequency;

[0064] Step S2: Scan the cache area at a third frequency to display the first image and / or the second image, wherein the third frequency is greater than or equal to the first frequency.

[0065] In this way, the first image set and the second image set are obtained by acquiring at the first frequency and the second frequency, and the processed first image information and the second image information are respectively stored in the cache area, and the cache area is scanned at a third frequency that is greater than or equal to the highest sampling frequency (i.e., the first frequency). The first image and / or the second image can be displayed in real time on the display module, ensuring that even if the sampling frequencies of the image sets are different, the display content can be refreshed at a sufficient frequency, thereby avoiding image freezes or display delays, while ensuring high image smoothness and real-time performance.

[0066] The first image set in step S1 includes a plurality of first image information, and the first image information may be the first image itself or an index value corresponding to the first image. In addition, the first image information may also include at least one of the format and size of the first image. In other words, the first image information includes at least one of the first image format, the first image size, the first image, and the index value corresponding to the first image.

[0067] Similarly, the second image set in step S1 includes a plurality of second image information, and the second image information may be the second image itself or an index value corresponding to the second image. In addition, the second image information may also include at least one of the format and size of the second image. In other words, the second image information includes at least one of the second image format, the second image size, the second image, and the index value corresponding to the second image.

[0068] Of course, the first image information and the second image information may also include other image data, such as grayscale images or color images, which may be adaptively increased or decreased according to actual applications, and the present invention does not impose specific limitations on this.

[0069] The first frequency in step S1 refers to the number of first image sets collected per unit time or the sampling rate of collecting the first image sets. The first frequency can be expressed in frames per second (FPS) or Hertz (Hz), which determines the update speed of the first image.

[0070] The second frequency in step S1 refers to the number of second image sets collected per unit time or the sampling rate of collecting the second image sets. Similar to the first frequency, the second frequency can also be expressed in frames per second (FPS) or Hertz (Hz). The second frequency determines the update speed of the second image.

[0071] The third frequency in step S2 refers to the rate at which the image display channel or the image display thread refreshes the image on the display module, and can also be measured in frames per second (FPS).

[0072] In one embodiment, the first frequency and the second frequency can be understood as sampling frequency or display frequency, which determines how many frames of images the image processing channel or image processing thread can output per second, thereby affecting the refresh rate and smoothness of the image on the display module.

[0073] Specifically, the first image is acquired at a first frequency, and the corresponding first image information is stored in a buffer at the same first frequency. The first image may come from a high frame rate image source, requiring a higher acquisition frequency to capture dynamic changes; and the second image is acquired by an image processing channel or an image processing thread at a second frequency, and the corresponding second image information is stored in a buffer at the same second frequency. The second image may come from a static or slowly changing image source, so a lower sampling frequency can be used.

[0074] In the above scheme, the first frequency is configured to be greater than the second frequency to limit the sampling frequency or display frequency of the two channels of images to be different; the third frequency is configured to be greater than the first frequency to ensure that the first image and the second image can be displayed on the display module in real time and smoothly. If the third frequency is lower than the first frequency, the image display channel may not be able to keep up with the output speed of the image processing channel, resulting in image freeze or delay.

[0075] In order to improve the image processing efficiency, the present invention can use two image processing channels for parallel processing to ensure that the first image set and the second image set can be processed synchronously and efficiently.

[0076] like Figure 4 As shown, in one embodiment, step S1 of the present invention may specifically include the following steps.

[0077] Step S11, sampling at a first frequency to obtain a first image set, processing the first image set by a first operation, and storing a plurality of first image information at the first frequency in a first cache area according to the order in which the images in the set are arranged;

[0078] Step S12, at the same time, sampling at a second frequency to obtain a second image set, processing the second image set with a second operation, and then storing a plurality of second image information at the second frequency in a second cache area according to the order in which the images in the set are arranged.

[0079] In this way, by simultaneously acquiring and processing two sets of images with different frequencies, the system can make full use of its processing capabilities to ensure that both types of image information can be stored in a timely and effective manner. This parallel processing method not only improves the response speed of the system, but also can more flexibly respond to different image acquisition and processing requirements, shorten the total time of image processing, and improve the throughput of image processing.

[0080] In one embodiment, the order in which the images in the set are arranged includes the time sequence in which the image set was acquired.

[0081] Among them, the first operation in step S11 is to perform an image processing operation on the first image, and the second operation in step S12 is to perform an image processing operation on the second image, and the image processing operation includes at least one of image enhancement, image transformation, image filtering, image segmentation and image recognition.

[0082] In addition, the processing of step S11 and step S12 is equivalent to the first image processing channel and the second image processing channel. Of course, this image processing channel can refer to the image processing flow (logical channel) realized by programming, or can refer to the channel (physical channel) actually existing in the image processing software. There is no specific limitation on this.

[0083] The first image processing channel refers to an independent path or module for processing the first image set; the second image processing channel refers to an independent path or module for processing the second image set. Of course, step S11 and step S12 can also be performed in sequence, and it may be that only in the display stage, the post-processed image may display the previous frame, while the first-processed image displays the current frame, but there will be no image display freeze.

[0084] Based on this, the sampling at the first frequency to obtain the first image set and processing the first image set by the first operation in step S11 include: using the first thread synchronization lock to control the synchronous triggering of the sampling process of the first image set and the first operation processing process.

[0085] In this way, the thread synchronization lock can ensure data consistency and operation sequence between the acquisition process and the image processing process of the first image set.

[0086] Thread synchronization lock, also known as synchronous thread lock, is used in multi-threaded programming to ensure that multiple threads do not conflict when accessing shared resources, thereby ensuring thread safety and data integrity.

[0087] In a specific embodiment, the sampling process of the first image set in step S11 can be implemented by writing a first image acquisition thread, and the first operation process of the first image set in step S11 can be implemented by writing a first image processing thread. In other words, the first image processing channel includes a first image acquisition thread (used to implement the acquisition of the first image set) and a first image processing thread (used to perform the first operation on the first image set).

[0088] Based on this, the first thread synchronization lock is used to control the synchronous triggering of the first image acquisition thread and the first image processing thread.

[0089] Specifically, before the first image acquisition thread performs an image acquisition operation, it is determined whether the first thread synchronization lock is held by other threads; if so, the first image acquisition thread is controlled to wait until the first thread synchronization lock is released; if not, the first thread synchronization lock is acquired, and the first image acquisition thread is controlled to acquire image data, and after the first image acquisition thread completes the image acquisition operation, the acquired first initial image is transmitted to the first image processing thread for processing.

[0090] Similarly, before the first image processing thread performs image processing operations, it determines whether the first thread synchronization lock is held by other threads; if so, the first image processing thread is controlled to wait until the first thread synchronization lock is released; if not, the first thread synchronization lock is acquired, and the first image processing thread is controlled to process the received first initial image.

[0091] Similarly, sampling at a second frequency to obtain a second image set and processing the second image set by a second operation include: using a second thread synchronization lock to control the synchronous triggering of the sampling process of the second image set and the second operation processing process.

[0092] In this way, the thread synchronization lock can ensure data consistency and operation sequence between the acquisition process and the image processing process of the second image set.

[0093] In a specific embodiment, the sampling process of the second image set in step S12 can be implemented by programming a second image acquisition thread, and the second operation process of the second image set in step S12 can be implemented by programming a second image processing thread. In other words, the second image processing channel includes a second image acquisition thread (used to implement the acquisition of the second image set) and a second image processing thread (used to perform the second operation on the second image set).

[0094] Based on this, the second thread synchronization lock is used to control the synchronous triggering of the second image acquisition thread and the second image processing thread.

[0095] Specifically, before the second image acquisition thread performs an image acquisition operation, it is determined whether the second thread synchronization lock is held by other threads; if so, the second image acquisition thread is controlled to wait until the second thread synchronization lock is released; if not, the second thread synchronization lock is acquired, and the second image acquisition thread is controlled to acquire image data, and after the second image acquisition thread completes the image acquisition operation, the acquired second initial image is transmitted to the second image processing thread for processing.

[0096] Similarly, before the second image processing thread performs image processing operations, it determines whether the second thread synchronization lock is held by other threads; if so, the second image processing thread is controlled to wait until the second thread synchronization lock is released; if not, the second thread synchronization lock is acquired, and the second image processing thread is controlled to process the received second initial image.

[0097] In a specific embodiment, the first image acquisition thread and the first image processing thread are executed in parallel; the second image processing channel includes a second image acquisition thread and a second image processing thread, and the second image acquisition thread and the second image processing thread are executed in parallel. In other words, the first image acquisition thread and the first image processing thread are independent of each other in execution, and the second image acquisition thread and the second image processing thread are also independent of each other in execution. The working process between the two can be referred to the following description, and will not be described in detail here.

[0098] In order to further improve the image processing capability, a multi-threaded pipeline operation is adopted based on the image processing channel or image processing thread. Specifically, a number of image threads are determined according to the image processing channel; an image processing task is obtained, decomposed into multiple subtasks, and the subtasks are assigned to the corresponding image threads according to the processing capability and task volume of each thread.

[0099] In one embodiment, the input data of the first image processing thread is determined according to the output data of the first image acquisition thread. Similarly, the input data of the second image processing thread is determined according to the output data of the second image acquisition thread.

[0100] Specifically, the image acquisition thread (including the first image acquisition thread and / or the second image acquisition thread) is responsible for acquiring image data from the image source (such as a camera, image file, etc.) and preparing it for subsequent processing. The work of this thread is the starting point of image processing. It continuously captures images from the source to ensure the real-time and integrity of the data.

[0101] The image processing thread (including the first image processing thread and / or the second image processing thread) relies on the output data of the image acquisition thread. After the image acquisition thread successfully acquires the image data, it passes the image data to the image processing thread. The image processing thread performs various processing on the received data, such as filtering, enhancement, recognition, etc., to generate the required image results.

[0102] The image acquisition thread and the image processing thread are interdependent, but they are independent in execution. The image acquisition thread can independently capture image data without waiting for the processing results of the image processing thread; similarly, the image processing thread can also independently process the received data without caring about the specific capture process of the image acquisition thread. This independence allows the two threads to work in parallel, thereby improving the efficiency of image processing.

[0103] From the perspective of the process, a pipeline-style working mode is formed between the image acquisition thread and the image processing thread. As the upstream of the pipeline, the image acquisition thread continuously transmits image data to the downstream (i.e., the image processing thread); while the image processing thread, as the downstream of the pipeline, processes the received data and outputs the final result. This pipeline-style working mode makes the image processing process more efficient and orderly.

[0104] In order to reduce the bandwidth of data transmission between the image acquisition thread and the image processing thread during the image processing process, the image data may be processed by transmitting the cache index corresponding to the image data. During the image display process, the corresponding display image is determined based on the cache index for display.

[0105] like Figure 5 As shown, in a specific embodiment, step S11 may specifically include the following steps.

[0106] Step S111, storing the N first images processed by the first operation into a first shared buffer, and storing N first index values ​​corresponding to the N first images into a first cache area at a first frequency; the first index value is used to index the corresponding first image for scanning and displaying the first image;

[0107] Likewise, step S12 may specifically include the following steps.

[0108] Step S121, store the M second images processed by the second operation into a second shared buffer, and store M second index values ​​corresponding to the M second images into a second cache area at a second frequency; the second index value is used to index the corresponding second image for scanning and displaying the second image.

[0109] In this way, by storing the image in the shared buffer and then storing the corresponding index value in the cache area at the same frequency, it can be ensured that the data in the cache is consistent with the original sampled data in time, which helps to accurately display the image data in the sampling order later.

[0110] Steps S111 and S121 can avoid unnecessary data caching and waiting time by matching the storage frequency and the sampling frequency, thereby improving the efficiency of the entire image processing process. For example, if the sampling frequency is high and the storage frequency is low, data in the cache may be accumulated, increasing processing delays; conversely, if the storage frequency is too high and the sampling frequency is low, storage space and processing resources may be wasted.

[0111] The N first images in step S111 may be generated after being processed by the first image processing thread, or may be generated by other processing methods. The N first images may be understood as part of the images in the first image set, or may be all the images in the first image set, and there is no specific limitation on this.

[0112] In addition, the shared buffer (including the first shared buffer and the second shared buffer) is used to store the first image itself, and the cache area (including the first cache area and the second cache area) is used to store the index value of the image ultimately used for display, that is, the image itself is stored at the cache address of the shared buffer.

[0113] In a specific embodiment, the first cache area is used to store the index value of the current first image, and the second cache area is used to store the index value of the current second image.

[0114] Among them, the current first image and the current second image refer to the images that are latest or most recently processed and ready for display at the current time point or processing cycle. These images are dynamically changing and may come from camera capture, image processing, image rendering and other processes, and will be replaced by new images over time.

[0115] The real-time cache index points to the storage address in the cache area of ​​the current first image, which means that when a new image is processed as the current first image, the index value will be updated in real time to point to the storage location of the new image. Similarly, the cache index value of the second image will also be updated in real time with the arrival of the new image.

[0116] like Figure 6 As shown, in another specific embodiment, step S11 may specifically include the following steps.

[0117] Step S112, storing the sampled first image set into a third shared buffer, and generating N third index values ​​corresponding to the N first images at a first frequency; the third index values ​​are used to index the corresponding first images for implementing the first operation;

[0118] Likewise, step S12 may specifically include the following steps.

[0119] Step S122, storing the sampled second image set into a fourth shared buffer, and generating M fourth index values ​​corresponding to the M second images at a second frequency; the fourth index values ​​are used to index the corresponding second images for implementing the second operation.

[0120] In this way, by storing the sampled first image set in the shared buffer and transferring the corresponding index value for implementing the first operation, the overhead caused by directly transferring a large amount of image data can be avoided, and the time and bandwidth required for data transmission can be reduced.

[0121] The third shared buffer points to the storage location of the first initial image obtained by sampling; after the first operation is performed on the first initial image, the first shared buffer points to the storage location of the first image generated after the first operation. The fourth shared buffer points to the storage location of the second initial image obtained by sampling; after the second operation is performed on the second initial image, the second shared buffer points to the storage location of the second image generated after the second operation.

[0122] In a specific embodiment, the first image processing channel may further include a first image rendering thread, which is used to update the first index value corresponding to the current first image to the first cache area.

[0123] In a specific embodiment, the second image processing channel may further include a second image rendering thread, which is used to update the second index value corresponding to the current second image to the second cache area.

[0124] For example, as shown in FIG7(a) and FIG7(b), assuming that the first image and the second image can be stored in a virtual memory (i.e., a shared buffer) that can be shared by multiple processes, the first image processing channel sequentially collects and processes a plurality of first image information in a time series, and stores them in a corresponding first buffer, and each buffer contains a unique index (i.e., an index value) of the buffer. Similarly, the second image processing channel sequentially collects and processes a plurality of second image information in a time series, and stores them in a corresponding second buffer.

[0125] As shown in Figures 7(a) and 7(b), the first image acquisition thread in the first image processing channel acquires a number of first initial images from the first video source at a rate of 60 frames per second (i.e., the first frequency), and writes the first initial images to the third shared buffer in the shared buffer, and passes the third index value of the third shared buffer to the first image processing thread; the first image processing thread receives and reads the corresponding first initial image from the third shared buffer according to the third index value, processes the first initial image, and rewrites the processed first image to the first shared buffer in the shared buffer, and passes the first index value of the first shared buffer to the first image rendering thread; the first image rendering thread receives the first index value and writes it to the first cache area (i.e., the first real-time image index cache area).

[0126] The second image processing channel is also processed in parallel. The second image acquisition thread acquires a number of second initial images from the second video source at a rate of 30 frames per second (i.e., the second frequency), writes the second initial images into a fourth shared buffer in the shared buffer, and passes the fourth index value of the fourth shared buffer to the second image processing thread; the second image processing thread receives and reads the corresponding second initial image from the fourth shared buffer according to the fourth index value, processes the second initial image, and rewrites the processed second image into the second shared buffer in the shared buffer, and passes the second index value of the second shared buffer to the second image rendering thread; the second image rendering thread receives the second index value and writes it into the second cache area (i.e., the second real-time image index cache area).

[0127] Control an image display channel or an image display process, such as at least one of a first display module process (corresponding to display module 1), a second display module process (corresponding to display module 2), and a third display module process (corresponding to display module 3), scan a first cache area and a second cache area at a third frequency, take out a current first index value and a second index value from the first cache area and the second cache area, query and determine the corresponding first image and second image from a shared buffer according to the first index value and the second index value, and display the first image and the second image according to a display mode of the display module.

[0128] As shown in FIG7(c), the sampling frequency of the first image is set to 60FPS (i.e., the first frequency), that is, the first image captures and / or refreshes a frame of the first image approximately every 16.67ms, and the sampling frequency of the second image is set to 30FPS (i.e., the second frequency), that is, the second image captures and / or refreshes a frame of the second image approximately every 33ms.

[0129] Specifically, when the first image acquisition thread acquires the first frame of the first image, the second image acquisition thread also acquires the first frame of the second image. At this time, the first cache area and the second cache area store the index values ​​of the first frame of the first image and the first frame of the second image respectively. The image display channel or the image display process scans the first buffer area and the second buffer area at a third frequency to display the first frame of the first image and the first frame of the second image; when the first image processing channel acquires and processes the second frame of the first image, the second image processing channel has not yet acquired and processed the second frame of the second image. At this time, the first cache area and the second cache area may store the index values ​​of the second frame of the first image and the first frame of the second image respectively. The image display channel or the image display process controls the display module to display the second frame of the first image and the first frame of the second image, that is, only the first image is refreshed and displayed, and the second image does not need to be refreshed, and the second image of the previous frame is directly retained, which is equivalent to only one image being displayed at this time to avoid jamming. By analogy, two images with different display frequencies can be displayed smoothly in the form of one frame of image.

[0130] In order to support the display operations of multiple display modules, a reference counter may be used to track and manage the allocation status of image resources, ensuring that image resources are used when appropriate and released in a timely manner when not needed.

[0131] In one embodiment, the first image set includes information of the current image, and / or the second image set includes information of the current image;

[0132] In one embodiment, the first image set includes information of the next image; and / or the second image information includes information of the next image.

[0133] In one embodiment, the first image set includes information of a current image, and the second image information includes information of a next image.

[0134] In one embodiment, the first image set includes information of a next image, and the second image information includes information of a current image.

[0135] In one embodiment, the first image set includes information of the next image, and the second image information includes information of the next image.

[0136] The current image refers to the image being displayed, processed or referenced at the current moment or the current processing stage. In an image sequence, if there is an image being viewed, rendered or waiting to be rendered to the screen, it is the current image. The next image refers to the image that follows the current image in the image sequence.

[0137] like Figure 8As shown, in one embodiment, scanning the cache area at a third frequency to display the first image and / or the second image may include the following steps.

[0138] Step S31, receiving a call instruction corresponding to P display modules, and scanning the cache area P times at a third frequency in response to the call instruction to display the current image until the next image is scanned and the display screen of the display module is updated with the next image, P≥1;

[0139] Step S32: when all P display modules display the next image, release the cache space corresponding to the currently displayed image in the cache area.

[0140] In this way, the current image is displayed by quickly scanning the cache area multiple times at a higher third frequency until a new image arrives and the display content is updated; and when all display modules display new images, the cache space of the original image is released to ensure that the display modules can display images quickly and continuously, while efficiently managing cache resources to avoid unnecessary space occupation.

[0141] like Fig. 9 As shown, in a specific embodiment, step S32 may specifically include the following steps.

[0142] Step S321, initializing a reference counting mark q corresponding to the current image, wherein the reference counting mark q is used to identify the number of times the current image is referenced;

[0143] Step S322, when receiving a call instruction from the display module for the current image, incrementing the reference count mark q by 1;

[0144] Step S323, when the display screen of the display module is updated to the next image, the reference count mark q is reduced by 1;

[0145] Step S324: according to the reference count mark q, release the cache space corresponding to the currently displayed image in the cache area.

[0146] In this way, by initializing a reference count mark to track the number of times the current image is referenced, it is possible to accurately grasp when the image to be displayed is no longer used by any module, thereby safely releasing the buffer resources occupied by it and avoiding memory leaks.

[0147] The reference count is an integer associated with an object, which is used to record the number of times the object is currently referenced. When an object is created, its reference count is initialized to 1 (indicating that the object itself has one reference). Whenever a new reference points to the object, the reference count increases by 1; and when a reference becomes invalid (for example, it is set to null or deleted), the reference count decreases by 1.

[0148] In a specific embodiment, step S324 may specifically include: when the counting mark q is reduced to 0, determining that all P display modules display the next image, and releasing the cache space corresponding to the currently displayed image in the cache area; wherein 0≤q≤P.

[0149] In this way, by detecting that the value of the reference count mark is reduced to 0, it means that the image is no longer displayed by any display module, and its memory can be safely released to avoid memory leaks.

[0150] One embodiment of the present invention provides a computer-readable storage medium.

[0151] In one embodiment, a computer readable storage medium stores a computer program executed by the processor mentioned above, or an image display method in any of the technical solutions mentioned above.

[0152] When the processor executes the computer program, it can execute the description of the image generation method in any of the above technical solutions, so it will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated.

[0153] The computer-readable storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0154] In summary, the image display method, system and storage medium provided by the present invention. In this way, by acquiring the first image set and the second image set at the first frequency and the second frequency, and storing the processed first image information and the second image information in the cache area respectively, and scanning the cache area at a third frequency greater than or equal to the highest sampling frequency (i.e., the first frequency), the first image and / or the second image can be displayed in real time on the display module, ensuring that even if the sampling frequencies of the image sets are different, the display content can be refreshed at a sufficient frequency, thereby avoiding the problem of image freeze or display delay, while ensuring the high smoothness and real-time performance of the image.

[0155] It should be understood that although this specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each implementation mode may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

[0156] The series of detailed descriptions listed above are only specific descriptions of feasible implementation methods of the present invention. They are not intended to limit the scope of protection of the present invention. Any equivalent implementation methods or changes that do not deviate from the technical spirit of the present invention should be included in the scope of protection of the present invention.

Claims

1. An image display method, characterized in that: include: Obtaining a first image set and a second image set, and storing processed first image information and second image information in a cache area respectively; The first image set is obtained by sampling at a first frequency, and the second image set is obtained by sampling at a second frequency, wherein the first frequency is greater than the second frequency; The cache area is scanned at a third frequency to display the first image and / or the second image, and the third frequency is greater than or equal to the first frequency.

2. The image display method according to claim 1, characterized in that: Obtaining a first image set and a second image set, and storing the processed first image set and the second image set in a cache area respectively, comprises: A first image set is obtained by sampling at a first frequency, and after processing the first image set by a first operation, a plurality of first image information is stored in a first cache area at the first frequency according to the order in which the images in the set are arranged; at the same time, A second image set is obtained by sampling at a second frequency, and after processing the second image set by a second operation, a plurality of second image information is stored in a second cache area at the second frequency according to the arrangement order of the images in the set.

3. The image display method according to claim 2, characterized in that: The method further comprises: Using the first thread synchronization lock, the sampling process of the first image set and the first operation processing process are controlled to be triggered synchronously; The second thread synchronization lock is used to control the synchronous triggering of the sampling process of the second image set and the second operation processing process.

4. The image display method according to claim 2, characterized in that: The method comprises: The N first images processed by the first operation are stored in a first shared buffer, and the N first index values ​​corresponding to the N first images are stored in a first cache area at a first frequency; the first index value is used to index the corresponding first image for scanning and displaying the first image; The M second images processed by the second operation are stored in a second shared buffer, and the M second index values ​​corresponding to the M second images are stored in a second cache area at a second frequency; the second index values ​​are used to index the corresponding second images for scanning and displaying the second images.

5. The image display method according to claim 2, characterized in that: The method comprises: The sampled first image set is stored in a third shared buffer, and N third index values ​​corresponding to the N first images are generated at a first frequency; the third index values ​​are used to index to the corresponding first image for implementing the first operation; The sampled second image set is stored in a fourth shared buffer, and M fourth index values ​​corresponding to the M second images are generated at a second frequency; the fourth index values ​​are used to index the corresponding second images for implementing the second operation.

6. The image display method according to claim 1, characterized in that: The step of scanning the cache area at a third frequency to display the first image and / or the second image includes: receiving a call instruction corresponding to P display modules, and scanning the cache area P times at a third frequency in response to the call instruction to display a current image until a next image is scanned and a display screen of the display module is updated with the next image, P≥1; When all P display modules display the next image, the buffer space corresponding to the currently displayed image in the buffer area is released.

7. The image display method according to claim 6, characterized in that: When all the P display modules display the next image, releasing the cache space corresponding to the currently displayed image in the cache area includes: Initialize a reference count mark q corresponding to the current image, wherein the reference count mark q is used to identify the number of times the current image is referenced; When receiving a call instruction for the current image from the display module, the reference count mark q is incremented by 1; When the display screen of the display module is updated to the next image, the reference count mark q is reduced by 1; According to the reference count mark q, the cache space corresponding to the currently displayed image in the cache area is released.

8. The image display method according to claim 7, characterized in that: The releasing the cache space corresponding to the currently displayed image in the cache area according to the reference count mark q comprises: When the counting mark q is reduced to 0, it is determined that all P display modules display the next image, and the cache space corresponding to the currently displayed image in the cache area is released; wherein 0≤q≤P.

9. The image display method according to claim 1, characterized in that: The method satisfies at least one of the following: The first image set includes information of the current image; and / or the second image set includes information of the current image; The first image set includes information of the next image; and / or the second image information includes information of the next image.

10. An image display system, characterized in that: include: A first processing module, configured to obtain a first image set by sampling at a first frequency, and store the processed first image information in a cache area; A second processing module, used for sampling at a second frequency to obtain a second image set, and storing the processed second image information in a cache area; A display module, configured to scan the cache area at a third frequency and display the first image and / or the second image; A buffer, used to establish the buffer area; The first frequency is greater than the second frequency, and the third frequency is greater than or equal to the first frequency.

11. The image display system according to claim 10, characterized in that: The first processing module includes a first sampling unit, a third shared buffer unit, a first processing unit, and a first shared buffer unit; the first sampling unit is used to sample at a first frequency to obtain a first image set; the third shared buffer unit is used to establish a third shared buffer, and the third shared buffer is used to store a third index value indexed to the first image to be processed; the first processing unit is used to perform a first operation; the first shared buffer unit is used to establish a first shared buffer, and the first shared buffer is used to store a first index value indexed to the processed first image. The second processing module includes a second sampling unit, a fourth shared buffer unit, a second processing unit, and a second shared buffer unit; the second sampling unit is used to sample at a second frequency to obtain a second image set; the fourth shared buffer unit is used to establish a fourth shared buffer, and the fourth shared buffer is used to store a fourth index value indexed to the second image to be processed; the second processing unit is used to perform a second operation, the second shared buffer unit is used to establish a second shared buffer, and the second shared buffer is used to store a second index value indexed to the processed second image.

12. A computer storage medium comprising: at least one processor; A memory storing a computer program executable on the processor, wherein the processor executes the steps of the image display method according to any one of claims 1 to 9 when executing the program.

Images