Image synchronization processing method, apparatus, device, storage medium, and program product

Abstract

The image synchronization processing method, device, equipment, storage medium and program product provided by the present application comprise the following steps: determining at least two target objects, and determining the change attribute of each target object; setting the image acquisition frame rate of the target object with the slowest change as a reference frame rate, and setting the image acquisition frame rate of other target objects on the basis of the reference frame rate, so that the image acquisition frame rates form a multiple relationship; and performing image acquisition according to the image acquisition frame rate of the target objects, and performing image synchronization processing on the basis of the reference frame rate. When at least two target objects are processed, the present application first determines the change attribute speed, sets the acquisition frame rate of the target object with the slowest change as a reference frame rate, and then determines the acquisition frame rate of other target objects on the basis of the reference frame rate. Finally, under the condition that the acquisition frame rates are different, the image data of the target objects is ensured to be synchronized, so as to reduce the processing quantity of the processor, reduce the processing and computing burden, and improve the overall efficiency of the processor.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image synchronization processing method, apparatus, device, storage medium, and program product. Background Technology

[0002] Currently, extended reality (XR) technology has gradually entered people's lives and various related technical fields. Corresponding augmented reality (AR) and virtual reality (VR) devices construct virtual stereoscopic visual environments by closing off human vision, allowing people to participate in the environment and achieve an immersive experience. Current extended reality devices generally collect images and movements of the user's body, and in order to coordinate the images of different parts of the body, these images are usually processed synchronously.

[0003] However, with the advancement of image processing sophistication, image sampling can be completed with a lower frame rate for slower-moving parts such as the head, but a higher frame rate is required for faster-moving parts such as the hands. Under the premise of synchronization, related technologies need to simultaneously increase or decrease the frame rates of both components, increasing the processor's processing burden and computational load, thus affecting the product's performance. Summary of the Invention

[0004] In view of this, this application proposes an image synchronization processing method, apparatus, device, storage medium, and program product, which reduces the number of processors, lowers the processing and computing burden, and improves the overall efficiency of the processor while ensuring the synchronization of these groups of image data.

[0005] To achieve the above objectives, this application provides an image synchronization processing method, comprising:

[0006] Identify at least two target objects and determine the variable attributes of each target object;

[0007] Based on the change attribute, the image acquisition frame rate of the target object with the slowest change is set as the baseline frame rate, and the image acquisition frame rate of other target objects is set based on the baseline frame rate, so that the image acquisition frame rates of each target object are in a multiple relationship.

[0008] Image acquisition is performed according to the image acquisition frame rate of each target object, and image synchronization processing is performed on the images corresponding to each target object based on the reference frame rate.

[0009] In some implementations, setting the image acquisition frame rate of other target objects based on the baseline frame rate includes:

[0010] The at least two target objects are sorted according to the rate of change of the change attribute, and the image acquisition frame rate of the other target objects is set to a multiple of the image acquisition frame rate of the target objects whose change is slower.

[0011] In some embodiments, the image synchronization processing of the images corresponding to each of the target objects includes:

[0012] Determine the ratio of the image acquisition frame rate to the reference frame rate for each target object;

[0013] The processing period is determined based on the baseline frame rate, and the processing period is divided according to the maximum value among the ratios.

[0014] The image corresponding to each target object is inserted into the divided processing cycle according to the corresponding ratio to perform image synchronization.

[0015] In some implementations, dividing the processing cycle according to the maximum value of the ratio includes:

[0016] Determine the number of the target objects;

[0017] The time within the processing cycle is divided into regions equal to the product of the maximum value of the ratio and the number of values.

[0018] In some implementations, inserting the image corresponding to each target object into the divided processing cycle according to the ratio includes:

[0019] For each target object, an equal number of regions within the processing cycle are selected sequentially at intervals;

[0020] Based on the ratio, a certain number of images corresponding to each target object are determined as images to be inserted.

[0021] Insert the image to be inserted into the corresponding divided region.

[0022] In some implementations, inserting the image to be inserted into the corresponding segmented region includes:

[0023] During the current processing cycle, if any of the target objects has completed the insertion of the image to be inserted, the partitioned regions corresponding to the target objects that have not yet had the image to be inserted are set to empty.

[0024] In some implementations, selecting equal-number division regions within the processing cycle for each target object at sequential intervals includes:

[0025] Arrange the at least two target objects sequentially according to the image acquisition frame rate corresponding to each target object;

[0026] The partitioned regions are selected sequentially at intervals within the processing cycle according to the arrangement order;

[0027] The step of inserting the image to be inserted into the corresponding segmented region includes:

[0028] According to the arrangement order, the corresponding images to be inserted are inserted sequentially and at intervals within the divided areas.

[0029] In some embodiments, inserting the images to be inserted into the divided regions sequentially at intervals includes:

[0030] Based on the image acquisition frame rate corresponding to each target object, the segmented region to be inserted for each target object is selected in the current processing cycle, so that the interval between the images to be inserted remains consistent across multiple processing cycles.

[0031] In some implementations, the images corresponding to each target object are arranged sequentially according to the acquisition time.

[0032] Based on the same concept, this application also provides an image synchronization processing apparatus, comprising:

[0033] A determination module is used to determine at least two target objects and to determine the variable attributes of each target object;

[0034] The setting module is used to set the image acquisition frame rate of the target object with the slowest change as the base frame rate according to the change attribute, and set the image acquisition frame rate of other target objects based on the base frame rate, so that the image acquisition frame rates of each target object form a multiple relationship.

[0035] The processing module is used to acquire images according to the image acquisition frame rate of each target object, and to perform image synchronization processing on the images corresponding to each target object based on the reference frame rate.

[0036] Based on the same concept, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method as described in any of the preceding claims.

[0037] Based on the same concept, this application also provides a non-transitory computer-readable storage medium that stores computer instructions for causing a computer to perform the method described in any of the preceding claims.

[0038] Based on the same concept, this application also provides a computer program product, including computer program instructions that, when run on a computer, cause the computer to perform the method described in any of the preceding claims.

[0039] As can be seen from the above description, the image synchronization processing method, apparatus, device, storage medium, and program product provided in this application include: determining at least two target objects and determining the change attributes of each target object; based on the change attributes, setting the image acquisition frame rate of the target object with the slowest change as a reference frame rate, and setting the image acquisition frame rates of other target objects based on the reference frame rate, so that the image acquisition frame rates of each target object form a multiple relationship; acquiring images according to the image acquisition frame rate of each target object, and performing image synchronization processing on the images corresponding to each target object based on the reference frame rate. When processing at least two target objects, this application first determines the speed of change attributes, sets the acquisition frame rate of the target object with the slowest change as a reference frame rate, and then determines the acquisition frame rates of other target objects accordingly. Finally, even with different acquisition frame rates, it ensures the synchronization of image data for these target objects, thereby reducing the number of processors, lowering the computational burden, and improving the overall efficiency of the processor. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 A flowchart illustrating an image synchronization processing method provided in an embodiment of this application;

[0042] Figure 2 This is a schematic diagram illustrating the synchronous processing of head images, hand images, and leg images at different frame rates in a specific embodiment provided for the present application.

[0043] Figure 3 This is a schematic diagram illustrating the synchronous processing of head and hand images at different frame rates in a specific embodiment provided in this application.

[0044] Figure 4 This is a schematic diagram of the structure of an image synchronization processing device provided in an embodiment of this application;

[0045] Figure 5 This is a schematic diagram of the electronic device structure provided in an embodiment of this application. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this specification clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0047] It is understood that before using the technical solutions disclosed in the various embodiments of this application, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this application in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0048] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this application's technical solution, based on the prompt message.

[0049] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0050] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this application. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this application.

[0051] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0052] To make the objectives, technical solutions, and advantages of this specification clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0053] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms "first," "second," and similar terms used in the embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element, object, or method step preceding the term covers the element, object, or method step listed after the term and its equivalents, without excluding other elements, objects, or method steps. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0054] As described in the background section, XR (Extended Reality) technology refers to an interactive environment that combines the real and virtual worlds through computer technology and wearable devices. XR technology can be considered a broad term encompassing VR (Virtual Reality), AR (Augmented Reality), MR (Mixed Reality), and other emerging immersive technologies. As a general term for immersive virtual and real-world fusion technologies, XR has evolved into different branches such as VR, AR, and MR. VR allows users to be completely immersed in a virtual environment; AR generates a world overlaid with virtual content but cannot interact with the real environment; MR is a hybrid of virtual and reality, creating virtual objects that can interact with the real environment. Currently, Extended Reality (XR) technology has gradually entered people's lives and various related technological fields. Augmented Reality (AR) and Virtual Reality (VR) technologies within XR have already been applied in shopping, gaming, entertainment, and fitness. Extended reality devices use multiple cameras to recognize the position of heads, hands, and gestures. For slower-moving objects like the head, a lower frame rate (e.g., 30fps) is sufficient to clearly capture head movements. For faster-moving objects like hands, a higher frame rate (e.g., 60fps) is generally required. However, to achieve synchronized processing, doubling the frame rate for the head and hands doubles the amount of data the main processor needs to process. For example, if the head and hands each need 30 frames per second, the main processor needs 60 frames per second. If this is increased to 60 frames per second each, the main processor needs to process 120 frames per second. But for slow-moving objects like the head, 30 frames per second is sufficient. This increases the processor's processing burden and computational load under the current synchronized processing conditions, impacting the product's usability.

[0055] In light of the above-mentioned practical situation, this application provides an image synchronization processing scheme. When processing at least two target objects, this application first determines the rate of change of attributes, sets the acquisition frame rate of the target object with the slowest change as the baseline frame rate, and then determines the acquisition frame rate of other target objects accordingly. Finally, even with different acquisition frame rates, the image data of these target objects is synchronized, thereby reducing the number of processors, lowering the computational burden, and improving the overall efficiency of the processor.

[0056] like Figure 1 The diagram shown is a flowchart illustrating an image synchronization processing method proposed in this application. Specifically, it includes:

[0057] Step 101: Identify at least two target objects and determine the variable attributes of each target object.

[0058] In some embodiments, the target object can be a target that needs to be image synchronized. They can be related to each other. For example, if an XR device needs to synchronize images of a user's head and hands, the target object is the user's head and hands. Alternatively, they can be unrelated to each other, but need to be synchronized in a specific scenario, such as synchronizing images of bicycles and cars on the same stretch of road.

[0059] Next, the change attributes of these target objects are determined. These change attributes can be attributes of overall or partial movement of a person or object, such as the positional movement of a head / handheld device / gesture, or the movement of a bicycle; they can also be attributes of changes in local features, such as changes in facial expressions or gestures. The specific methods of these changes may vary, but each change corresponds to an attribute of its rate of change, which can be understood as the speed of change. This speed of change can be generated through comparisons between objects or with a baseline value. For example, comparing the user's head and hand, the speed of change of the head is significantly less than that of the hand. Alternatively, the movement speed can be calculated separately to determine the relative speeds of change for each object. All of these can be measured and calculated using sensors. Alternatively, a table recording the speed of change can be set up based on historical experience to record the speed of change for some common objects. The speed of change here is generally relative, meaning that comparisons are made between the target objects that require image synchronization to determine their relative speeds of change.

[0060] Step 102: Based on the change attribute, set the image acquisition frame rate of the target object with the slowest change as the base frame rate, and set the image acquisition frame rate of other target objects based on the base frame rate, so that the image acquisition frame rates of each target object form a multiple relationship.

[0061] In some embodiments, the image acquisition frame rate is generally expressed as the frame rate data during image acquisition. Frame rate data (FPS, Frames Per Second) simply refers to the number of frames transmitted per second in an animation or video. For example, 30fps means acquiring 30 images per second for processing and display. The baseline frame rate can be a pre-set frame rate value, which can be specifically set according to different scenarios or different facial objects. For example, in indoor activity scenarios, images with slow-moving speeds, such as user heads, only require 30fps to clearly reflect the corresponding image; while in other scenarios, such as image capture on a highway, although bicycles are relatively slow-moving images, 60fps is still needed to clearly reflect the corresponding image.

[0062] Next, the image capture frame rate of the target objects is set to a multiple of the target objects. For example, for three target objects—head, hand, and leg—the head changes the slowest, followed by the leg, and then the hand. Therefore, the image capture frame rate of the head is set to a base frame rate, such as 30fps. Then, the leg's frame rate is set to a multiple of the head's, such as twice the head's, resulting in a leg image capture frame rate of 60fps. Similarly, the hand can be a multiple of both the leg and head's frame rates, so the hand's frame rate could be four times that of the head and twice that of the leg, resulting in a hand image capture frame rate of 120fps. Of course, the specific multiples can be set according to the specific scenario.

[0063] Step 103: Image acquisition is performed according to the image acquisition frame rate of each target object, and image synchronization processing is performed on the images corresponding to each target object based on the reference frame rate.

[0064] In some embodiments, images of the target objects are acquired according to a determined image acquisition frame rate for each target object. Each set of target object images can be image data acquired per second, for example, a set of image data for the head (30fps) consists of 30 head images acquired per second, a set of image data for the hands (60fps) consists of 60 hand images acquired per second, and so on. The images in each set can be arranged according to their acquisition time. Since a multiple relationship between the image acquisition frame rates of the target objects has been set in step 102, the number of images acquired here is related to their image acquisition frame rate. For example, if the frame rate for head images is 30fps, then 30 images are acquired per second; if the frame rate for hand images is 120fps, then 120 images are acquired per second; if the frame rate for leg images is 60fps, then 60 images are acquired per second, and so on.

[0065] Subsequently, based on the baseline frame rate, image synchronization processing is performed on the images corresponding to each target object. To ensure synchronization, it is necessary to ensure that at least one image from each group of image data is processed within a processing cycle. Therefore, based on the baseline frame rate, it is crucial to ensure that at least one image of the slowest target object is processed within a cycle. For example, if the head has the lowest changing speed among all target objects, with a baseline frame rate of 30fps, corresponding to processing one image every 1 / 30s, then according to this scheme, the processing cycle is 1 / 30s, and one head image can be processed within one cycle. Furthermore, since the acquisition frame rates of other target objects are multiples of those of the target object with the lowest changing frame rate, a cycle can be divided according to this multiple, and images of the corresponding multiple of the target objects can be processed within one cycle, thus achieving synchronization.

[0066] In one specific embodiment, taking a frame rate of 30fps for head images, 120fps for hand images, and 60fps for leg images as an example, the multiple between the hand and head is 4, the multiple between the leg and head is 2, and the multiple between the hand and leg is 2. The processing cycle is divided based on the maximum value of 4. This division can be based on a specific multiple of this maximum value. For example, if there are 3 sets of data, the processing cycle is divided into 12 parts, which are the product of the maximum value of 4 and the number of sets of 3. That is, a 1 / 30s processing cycle is divided into 12 parts, each a 1 / 360s segment. Figure 2 As shown. Then, one image is selected from the head image data based on the magnification factor, four images are selected from the hand image data based on the magnification factor, and two images are selected from the leg image data based on the magnification factor. These are then placed into the predefined processing cycles, as follows: Figure 2 As shown, this ensures the synchronization between different groups of image data while maintaining the processing frame rate data for each group of image data.

[0067] In another specific embodiment, according to the scheme of the foregoing embodiment, the image acquisition frame rate is 30fps for head images and 60fps for hand images, such as... Figure 3 As shown, the processing cycle is 1 / 30s. Dividing the processing cycle according to the maximum value of the multiplier (2), since there are two sets of data, the processing cycle is divided into four parts, each 1 / 120s. Then, based on the multiplier, one image is selected from the head image and two images from the hand image, and these are placed into the divided processing cycles. In this way, while maintaining synchronization, the image processing frame rate only needs to be 90 frames per second. Compared to related solutions that require simultaneous changes in both frames (either both at 30 frames or both at 60 frames), this reduces the computational load by 30 frames, reduces the number of processors, lowers the computational burden, and improves the overall processor efficiency.

[0068] Ultimately, the system can output the completed processing cycle, which can then be used as a basis for further image processing. It also serves to store, display, use, or reprocess this processing cycle. The specific output method for this processing cycle can be flexibly selected according to different application scenarios and implementation needs.

[0069] For example, in the application scenario where the method of this embodiment is executed on a single device, the processing cycle can be directly applied to the image processing of the current device and output in an execution manner, so that the image display of the current device is processed and displayed according to the processing cycle.

[0070] For example, in application scenarios where the method of this embodiment is executed on a system composed of multiple devices, the processing cycle can be sent to other preset devices within the system as recipients, i.e., synchronization terminals, via any data communication method (wired connection, NFC, Bluetooth, Wi-Fi, cellular mobile network, etc.), so that the synchronization terminals can perform subsequent processing. Optionally, the synchronization terminal can be a preset server, which is generally located in the cloud and serves as a data processing and storage center, capable of storing and distributing the processing cycle; wherein, the recipients of the distribution are terminal devices, and the owners or operators of these terminal devices can be other smart devices of the current user, maintainers of smart devices, manufacturers of related smart devices, etc.

[0071] For example, in the application scenario where the method of this embodiment is executed on a system composed of multiple devices, the processing cycle can be directly sent to a preset terminal device through any data communication method. The terminal device can be one or more of the devices listed in the preceding paragraphs.

[0072] As described above, the image synchronization processing method provided in this application includes: determining at least two target objects and determining the change attributes of each target object; based on the change attributes, setting the image acquisition frame rate of the target object with the slowest change as a reference frame rate, and setting the image acquisition frame rates of other target objects based on the reference frame rate, so that the image acquisition frame rates of each target object form a multiple relationship; acquiring images according to the image acquisition frame rate of each target object, and performing image synchronization processing on the images corresponding to each target object based on the reference frame rate. When processing at least two target objects, this application first determines the speed of change attributes, sets the acquisition frame rate of the target object with the slowest change as a reference frame rate, and then determines the acquisition frame rates of other target objects accordingly. Finally, even with different acquisition frame rates, it ensures the synchronization of image data for these target objects, thereby reducing the number of processor operations, lowering the computational burden, and improving the overall efficiency of the processor.

[0073] In some embodiments, setting the image acquisition frame rate of other target objects based on the baseline frame rate includes: sorting the at least two target objects according to the rate of change of the change attribute, and setting the image acquisition frame rate of the other target objects to a set multiple of the image acquisition frame rate of the target objects whose change is slower.

[0074] In some embodiments, to quickly set the image acquisition frame rate for each target object, they can be sorted according to the rate of change of different target objects' change attributes. The sorting can be from fast to slow or from slow to fast, with the slowest changing image acquisition frame rate set as the base frame rate. The second-to-last frame rate is set as a multiple of the base frame rate, the third-to-last as a multiple of the second-to-last, and so on, ensuring that the image acquisition frame rates of each target object are in a multiple relationship, thus facilitating subsequent image synchronization processing. Furthermore, in this embodiment, the image acquisition frame rate corresponding to a fast-changing target object is in a multiple relationship with the image acquisition frame rate corresponding to a target object with a slower change. For example, the frame rate for a head image is 30fps, the frame rate for a hand image is 120fps, and the frame rate for a leg image is 60fps. The frame rate of the hand image is four times that of the head image and twice that of the leg image. It can be seen that if a target object corresponds to multiple other target objects, the set multiples can also be multiple.

[0075] In some embodiments, the image synchronization processing of the images corresponding to each of the target objects includes: determining the ratio between the image acquisition frame rate of each target object and the reference frame rate; determining a processing period based on the reference frame rate; dividing the processing period according to the maximum value of the ratio; and inserting the image corresponding to each target object into the divided processing period according to the corresponding ratio to perform the image synchronization.

[0076] In some embodiments, in order to ensure synchronization requirements, it is necessary to determine that at least one image of each target object is processed within a processing cycle. Here, the processing cycle can be based on the processing time corresponding to the baseline frame rate. For example, if the baseline frame rate is 30fps, which corresponds to processing one image every 1 / 30s, then according to this scheme, the processing cycle is 1 / 30s.

[0077] Next, the processing cycle is determined based on the baseline frame rate, and the processing cycle is divided according to the maximum value among the ratios. For example, if the frame rate of the head image is 30fps, the frame rate of the hand image is 120fps, and the frame rate of the leg image is 60fps, then the ratio of the hand to the head is 4, the ratio of the leg to the head is 2, and the ratio of the hand to the leg is 2. The processing cycle is then divided according to the maximum value of 4. The division method can be based on a specific multiple of this maximum value. For example, if there are 3 sets of data, the processing cycle is divided into 12 parts, which is the product of the maximum value of 4 and the number of sets of 3. That is, the 1 / 30s processing cycle is divided into 12 divisions of 1 / 360s each. Figure 2 As shown.

[0078] Finally, the image corresponding to each target object is inserted into the divided processing cycle according to the ratio to perform image synchronization. Following the previous example, one image is selected from the head image data based on the ratio, four images are selected from the hand image data based on the ratio, and two images are selected from the leg image data based on the ratio, and placed into the divided processing cycle, such as... Figure 2 As shown, this ensures the synchronization between different groups of image data while maintaining the processing frame rate data for each group of image data.

[0079] In some embodiments, dividing the processing cycle according to the maximum value of the ratio includes: determining the number of target objects; and dividing the time within the processing cycle into a division region consisting of the product of the maximum value of the ratio and the number of objects.

[0080] In some embodiments, to ensure that the frame rate of each group of image data remains synchronized, the frame rate requirements of each target object with a high frame rate need to be met when dividing the processing cycle. This allows the specific number of target objects to be determined; for example, the head and hands are considered two target objects, while the head, hands, and legs are considered three target objects. This number is then multiplied by the maximum value in the ratio to determine the number of regions to be divided; these divided regions are the segmented regions. For example, as... Figure 2 As shown, there are 3 sets of data here: the frame rate of the head image is 30fps, the frame rate of the hand image is 120fps, and the frame rate of the leg image is 60fps. The processing cycle is divided into 12 parts by multiplying the maximum value of the frame rate ratio (4) by the number of groups (3). That is, the 1 / 30s processing cycle is divided into 12 regions of 1 / 360s each.

[0081] In some embodiments, inserting the image corresponding to each target object into the divided processing cycle according to the ratio includes: selecting an equal number of division regions within the processing cycle for each target object at intervals; determining a corresponding number of images to be inserted from the images corresponding to each target object according to the ratio; and inserting the images to be inserted into the corresponding division regions.

[0082] In some embodiments, such as Figure 2 or Figure 3 As shown, a considerable number of partitioned regions can be selected for each target object. Since the partitioned regions are determined by the product of the number of groups and the maximum value among the ratios, each target object will necessarily have partitioned regions containing the maximum number of ratio values. Figure 2 In the image, the head, hand, and leg images each correspond to four segmented regions; for example... Figure 3 In this process, both the head and hand images correspond to two segmented regions. Then, based on the ratio, the number of images to be inserted for each target object in one cycle is determined; these images are the images to be inserted. For example... Figure 2 In this example, a head image requires one image to be inserted, a hand image requires four images to be inserted, and a leg image requires two images to be inserted; for example... Figure 3 In the process, one image needs to be inserted for the head image and two images need to be inserted for the hand image. Finally, the selected images are inserted into the corresponding segmented regions. Only one image is inserted into each segmented region.

[0083] In some embodiments, inserting the image to be inserted into the corresponding partitioned region includes: within the current processing cycle, if any of the target objects has already completed the insertion of the image to be inserted, setting the other partitioned regions corresponding to that target object that have not yet inserted the image to be inserted to empty.

[0084] In some embodiments, such as Figure 2 or Figure 3 As shown, if any set of image data has already completed image insertion within this processing cycle, then the corresponding regions of other images that have not yet been inserted are set to empty. For example... Figure 2 In this process, the head image corresponds to four partitioned regions, but there is only one image to be inserted. After inserting one partitioned region into this one image, the other partitioned regions are left empty. The order or pattern of image insertion in each processing cycle is not specifically limited; it only requires that the corresponding number of images are inserted within one processing cycle.

[0085] In some embodiments, selecting equal number of partitioned regions within the processing cycle for each target object at intervals includes: sequentially arranging the at least two target objects according to the image acquisition frame rate corresponding to each target object; sequentially selecting the partitioned regions at intervals within the processing cycle according to the arrangement order; and inserting the image to be inserted into the corresponding partitioned region includes: sequentially inserting the corresponding image to be inserted into the partitioned region at intervals according to the arrangement order.

[0086] In some embodiments, to facilitate the management of images of various target objects, the images can first be sorted according to the frame rate of each target object, and then the regions can be selected sequentially according to the sorting order. For example, if the frame rate of the head image is 30fps, the frame rate of the hand image is 120fps, and the frame rate of the leg image is 60fps, then the images can be sorted as head image, leg image, and hand image. When selecting the regions, the first region is the head image region, the second region is the leg image region, the third region is the hand image region, the fourth region is the head image region, and so on. Therefore, when inserting images, the images to be inserted are also inserted according to the sorting order.

[0087] In some embodiments, inserting the images to be inserted into the divided regions sequentially and at intervals includes: selecting the divided regions where the images to be inserted for each target object need to be inserted within the current processing cycle, based on the image acquisition frame rate corresponding to each target object, so that the intervals between the images to be inserted remain consistent across multiple processing cycles.

[0088] In some embodiments, similar to the foregoing embodiments, if any set of image data has completed image insertion within a processing cycle, then the partitioned regions corresponding to that set of image data that have not yet had images inserted are set to empty. In this embodiment, since each target object has a corresponding image acquisition frame rate, to facilitate processing and image insertion, the number of insertions within a processing cycle can be guaranteed while ensuring that the image acquisition interval for the same target object remains consistent between processing cycles. This ensures that image processing at different frame rates can be completed quickly without frame rate adjustment. (See attached...) Figure 2 As shown, the frame rate of the head image is 30fps, the frame rate of the hand image is 120fps, and the frame rate of the leg image is 60fps. The spacing between the head image, hand image, and leg image is also the same between processing cycles. In this way, within one second, the frame rate of the head image is still 30fps, the frame rate of the hand image is still 120fps, and the frame rate of the leg image is still 60fps.

[0089] In some embodiments, the images corresponding to each target object are arranged sequentially according to the acquisition time.

[0090] It should be noted that the method in this application embodiment can be executed by a single device, such as a computer or server. The method in this application embodiment can also be applied in a distributed scenario, where multiple devices cooperate to complete the process. In such a distributed scenario, one of these devices may execute only one or more steps of the method in this application embodiment, and the multiple devices will interact with each other to complete the method described.

[0091] It should be noted that the above description describes specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0092] Based on the same concept, corresponding to any of the above embodiments, this application also provides an image synchronization processing apparatus.

[0093] refer to Figure 4 The image synchronization processing device includes:

[0094] The determination module 210 is used to determine at least two target objects and determine the change attributes of each target object;

[0095] The setting module 220 is used to set the image acquisition frame rate of the target object with the slowest change as the base frame rate according to the change attribute, and set the image acquisition frame rate of other target objects based on the base frame rate, so that the image acquisition frame rates of each target object form a multiple relationship.

[0096] The processing module 230 is used to acquire images according to the image acquisition frame rate of each target object, and to perform image synchronization processing on the images corresponding to each target object based on the reference frame rate.

[0097] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing the embodiments of this application, the functions of each module can be implemented in one or more software and / or hardware.

[0098] The apparatus described above is used to implement the corresponding image synchronization processing method in the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0099] In some embodiments, the setting module 220 is further configured to:

[0100] The at least two target objects are sorted according to the rate of change of the change attribute, and the image acquisition frame rate of the other target objects is set to a multiple of the image acquisition frame rate of the target objects whose change is slower.

[0101] In some embodiments, the processing module 230 is further configured to:

[0102] Determine the ratio of the image acquisition frame rate to the reference frame rate for each target object;

[0103] The processing period is determined based on the baseline frame rate, and the processing period is divided according to the maximum value among the ratios.

[0104] The image corresponding to each target object is inserted into the divided processing cycle according to the corresponding ratio to perform image synchronization.

[0105] In some embodiments, the processing module 230 is further configured to:

[0106] Determine the number of the target objects;

[0107] The time within the processing cycle is divided into regions equal to the product of the maximum value of the ratio and the number of values.

[0108] In some embodiments, the processing module 230 is further configured to:

[0109] For each target object, an equal number of regions within the processing cycle are selected sequentially at intervals;

[0110] Based on the ratio, a certain number of images corresponding to each target object are determined as images to be inserted.

[0111] Insert the image to be inserted into the corresponding divided region.

[0112] In some embodiments, the processing module 230 is further configured to:

[0113] During the current processing cycle, if any of the target objects has completed the insertion of the image to be inserted, the partitioned regions corresponding to the target objects that have not yet had the image to be inserted are set to empty.

[0114] In some embodiments, the processing module 230 is further configured to:

[0115] Arrange the at least two target objects sequentially according to the image acquisition frame rate corresponding to each target object;

[0116] The partitioned regions are selected sequentially at intervals within the processing cycle according to the arrangement order;

[0117] The processing module 230 is further configured to:

[0118] According to the arrangement order, the corresponding images to be inserted are inserted sequentially and at intervals within the divided areas.

[0119] In some embodiments, the processing module 230 is further configured to:

[0120] Based on the image acquisition frame rate corresponding to each target object, the segmented region to be inserted for each target object is selected in the current processing cycle, so that the interval between the images to be inserted remains consistent across multiple processing cycles.

[0121] In some embodiments, the images corresponding to each target object are arranged sequentially according to the acquisition time.

[0122] Based on the same concept, corresponding to the methods of any of the above embodiments, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the image synchronization processing method as described in any of the above embodiments.

[0123] Figure 5 This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0124] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0125] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0126] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0127] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0128] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0129] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0130] The electronic devices described above are used to implement the corresponding image synchronization processing methods in any of the foregoing embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0131] Based on the same concept, corresponding to the methods of any of the above embodiments, this application also provides a non-transitory computer-readable storage medium that stores computer instructions for causing the computer to execute the image synchronization processing method as described in any of the above embodiments.

[0132] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0133] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the image synchronization processing method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0134] Based on the same concept, corresponding to any of the above-described embodiments, this application also provides a computer program product, which includes computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer to cause the computer and / or the processor to perform the image synchronization processing method. Corresponding to the execution entity for each step in each embodiment of the image synchronization processing method, the processor executing the corresponding step may belong to the corresponding execution entity.

[0135] The computer program product of the above embodiments is used to cause the computer and / or the processor to execute the image synchronization processing method as described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0136] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application (including the claims) is limited to these examples; within the framework of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in the details for the sake of brevity.

[0137] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this application, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0138] Although this application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0139] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this application.

Claims

1. An image synchronization processing method, characterized in that, include: Identify at least two target objects and determine the variable attributes of each target object; Based on the change attribute, the image acquisition frame rate of the target object with the slowest change is set as the baseline frame rate, and the image acquisition frame rates of other target objects are set based on the baseline frame rate, so that the image acquisition frame rates of each target object form a multiple relationship; the image acquisition frame rate is the frame rate data for image acquisition. Image acquisition is performed according to the image acquisition frame rate of each target object, and image synchronization processing is performed on the images corresponding to each target object based on the reference frame rate; wherein, by using the reference frame rate as a basis, it is ensured that the target object with the lowest frame rate has one image processed in one cycle, and the number of images processed in one cycle for other target objects is determined according to the multiple relationship. The step of performing image synchronization processing on the images corresponding to each of the target objects includes: determining the ratio between the image acquisition frame rate and the reference frame rate for each target object; determining a processing period based on the reference frame rate; dividing the processing period according to the maximum value of the ratio; and inserting the image corresponding to each target object into the divided processing period according to the corresponding ratio to perform the image synchronization.

2. The method according to claim 1, characterized in that, Setting the image acquisition frame rate of other target objects based on the baseline frame rate includes: The at least two target objects are sorted according to the rate of change of the change attribute, and the image acquisition frame rate of the other target objects is set to a multiple of the image acquisition frame rate of the target objects whose change is slower.

3. The method according to claim 1, characterized in that, The step of dividing the processing cycle according to the maximum value of the ratio includes: Determine the number of the target objects; The time within the processing cycle is divided into regions equal to the product of the maximum value of the ratio and the number of values.

4. The method according to claim 3, characterized in that, The step of inserting the image corresponding to each target object into the divided processing cycle according to the ratio includes: For each target object, an equal number of regions within the processing cycle are selected sequentially at intervals; Based on the ratio, a certain number of images corresponding to each target object are determined as images to be inserted. Insert the image to be inserted into the corresponding divided region.

5. The method according to claim 4, characterized in that, The step of inserting the image to be inserted into the corresponding segmented region includes: During the current processing cycle, if any of the target objects has completed the insertion of the image to be inserted, the partitioned regions corresponding to the target objects that have not yet had the image to be inserted are set to empty.

6. The method according to claim 4, characterized in that, The step of sequentially selecting equal-number division regions within the processing cycle for each target object includes: Arrange the at least two target objects sequentially according to the image acquisition frame rate corresponding to each target object; The partitioned regions are selected sequentially at intervals within the processing cycle according to the arrangement order; The step of inserting the image to be inserted into the corresponding segmented region includes: According to the arrangement order, the corresponding images to be inserted are inserted sequentially and at intervals within the divided areas.

7. The method according to claim 6, characterized in that, The sequential, intermittent insertion of the images to be inserted into the divided regions includes: Based on the image acquisition frame rate corresponding to each target object, the segmented region to be inserted for each target object is selected in the current processing cycle, so that the interval between the images to be inserted remains consistent across multiple processing cycles.

8. The method according to claim 1, characterized in that, The images corresponding to each target object are arranged in order according to the acquisition time.

9. An image synchronization processing device, characterized in that, include: A determination module is used to determine at least two target objects and to determine the variable attributes of each target object; The setting module is used to set the image acquisition frame rate of the target object with the slowest change as the base frame rate according to the change attribute, and set the image acquisition frame rate of other target objects based on the base frame rate, so that the image acquisition frame rates of each target object form a multiple relationship; the image acquisition frame rate is the frame rate data for image acquisition. The processing module is used to acquire images according to the image acquisition frame rate of each target object, and to perform image synchronization processing on the images corresponding to each target object based on the reference frame rate; wherein, by using the reference frame rate as a basis, it is ensured that the target object with the lowest frame rate has one image processed in one cycle, and the number of images processed in one cycle for other target objects is determined according to the multiple relationship. The processing module is further configured to determine the ratio between the image acquisition frame rate and the reference frame rate for each target object; determine the processing period based on the reference frame rate; divide the processing period according to the maximum value of the ratio; and insert the image corresponding to each target object into the divided processing period according to the corresponding ratio to perform image synchronization.

10. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method as described in any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions for causing a computer to perform the method according to any one of claims 1 to 8.

12. A computer program product, characterized in that, It includes computer program instructions that, when run on a computer, cause the computer to perform the method as described in any one of claims 1 to 8.

Images