Target image group determination method and device, equipment, storage medium and vehicle

Abstract

The invention discloses a target image group determination method and device, equipment, a storage medium and a vehicle. The method comprises the steps that a first image sequence of a first camera and timestamps of all images in the first image sequence are obtained, a second image sequence of a second camera and timestamps of all images in the second image sequence are obtained, and the first camera and the second camera are cameras arranged at different shooting angles of a vehicle; comparing the timestamps of the images in the first image sequence with the timestamps of the images in the second image sequence to obtain a comparison result; and when the comparison result indicates that the time stamp interval between the first image in the first image sequence and the second image in the second image sequence is smaller than or equal to the preset threshold value, determining the first image and the second image as an aligned target image group.

Description

Technical Field

[0001] The present application belongs to the field of vehicle technology, and in particular, relates to a method, device, equipment, storage medium and vehicle for determining a target image group. Background Art

[0002] With the continuous development of vehicle technology, its intelligence level is getting higher and higher. In the existing vehicle intelligent space interaction process, the in-vehicle image can be collected by at least two cameras in different directions in the vehicle cabin, and input into the recognition model to identify the interaction information such as the gestures or specific positions of the people in the car, thereby realizing the intelligent interaction between man and machine. Before the in-vehicle image is input into the recognition model to recognize the interaction information, it is necessary to align the in-vehicle images collected by at least two cameras in different directions, but the image alignment accuracy of the existing technology is low, resulting in deviations in the output results of the recognition model. Summary of the invention

[0003] The embodiments of the present application provide a method, apparatus, device, storage medium and vehicle for determining a target image group, which can realize the alignment of images captured by cameras at different shooting angles in a vehicle, thereby improving the accuracy of the vehicle's intelligent interaction model in identifying interaction information.

[0004] In a first aspect, an embodiment of the present application provides a method for determining a target image group, the method comprising:

[0005] Acquire a first image sequence from a first camera and a timestamp of each image in the first image sequence, and acquire a second image sequence from a second camera and a timestamp of each image in the second image sequence, wherein the first camera and the second camera are cameras set at different shooting angles of the vehicle;

[0006] comparing the time stamps of the images in the first image sequence with the time stamps of the images in the second image sequence to obtain a comparison result;

[0007] When the comparison result indicates that the time stamp interval between a first image in the first image sequence and a second image in the second image sequence is less than or equal to a preset threshold, the first image and the second image are determined as an aligned target image group.

[0008] In a second aspect, an embodiment of the present application provides a device for determining a target image group, the device comprising:

[0009] an acquisition module, configured to acquire a first image sequence from a first camera and a timestamp of each image in the first image sequence, and to acquire a second image sequence from a second camera and a timestamp of each image in the second image sequence, wherein the first camera and the second camera are cameras set at different shooting angles of the vehicle;

[0010] a comparison module, configured to compare the time stamps of the images in the first image sequence with the time stamps of the images in the second image sequence to obtain a comparison result;

[0011] A determination module is used to determine the first image and the second image as an aligned target image group if the comparison result indicates that the timestamp interval between the first image in the first image sequence and the second image in the second image sequence is less than or equal to a preset threshold.

[0012] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the method for determining a target image group as described in any one of the above items is implemented.

[0013] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, a method for determining a target image group as described in any one of the above items is implemented.

[0014] In a fifth aspect, an embodiment of the present application provides a computer program product. When instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes a method for determining a target image group as described in any one of the above.

[0015] In a sixth aspect, an embodiment of the present application provides a vehicle, comprising: an electronic device, wherein the electronic device is used to implement the method for determining a target image group as described in any one of the above items.

[0016] The method, device, equipment, storage medium and vehicle for determining the target image group of the embodiment of the present application can obtain the first image sequence of the first camera and the timestamp of each image in the first image sequence, and obtain the second image sequence of the second camera and the timestamp of each image in the second image sequence, the first camera and the second camera are cameras set at different shooting angles of the vehicle; compare the timestamps of the images in the first image sequence with the timestamps of the images in the second image sequence to obtain a comparison result; finally, when the comparison result indicates that the timestamp interval between the first image in the first image sequence and the second image in the second image sequence is less than or equal to a preset threshold, the first image and the second image are determined as the aligned target image group. In this way, the embodiment determines the first image and the second image whose timestamp interval is less than or equal to the preset threshold as the aligned target image group by comparing the timestamp interval between the timestamp of the first image in the first image sequence and the timestamp of the second image in the second image sequence, so as to realize the alignment of images collected by cameras at different shooting angles in the vehicle, thereby improving the accuracy of the vehicle intelligent interaction model in identifying interaction information. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0018] Figure 1 is a flowchart of a method for determining a target image group provided by an embodiment of the present application;

[0019] Figure 2 is a schematic structural diagram of a device for determining a target image group provided by another embodiment of the present application;

[0020] Figure 3 It is a structural diagram of an electronic device provided by yet another embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0022] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0023] With the continuous development of vehicle technology, its intelligence level is getting higher and higher. In the existing vehicle intelligent space interaction process, the in-vehicle image can be collected by at least two cameras in different directions in the vehicle cabin, and input into the recognition model to identify the interaction information such as the gestures or specific positions of the people in the car, thereby realizing the intelligent interaction between man and machine. Before the in-vehicle image is input into the recognition model to recognize the interaction information, it is necessary to align the in-vehicle images collected by at least two cameras in different directions, but the image alignment accuracy of the existing technology is low, resulting in deviations in the output results of the recognition model.

[0024] In order to solve the problems in the prior art, the embodiments of the present application provide a method, device, equipment, storage medium and vehicle for determining a target image group. The method for determining a target image group provided by the embodiments of the present application is first introduced below.

[0025] Figure 1 FIG. 1 is a flow chart showing a method for determining a target image group provided by an embodiment of the present application. Figure 1 As shown, a method for determining a target image group may include the following steps S101 to S103:

[0026] S101, obtaining a first image sequence from a first camera and a timestamp of each image in the first image sequence, and obtaining a second image sequence from a second camera and a timestamp of each image in the second image sequence, wherein the first camera and the second camera are cameras set at different shooting angles of the vehicle;

[0027] S102, comparing the time stamps of the images in the first image sequence with the time stamps of the images in the second image sequence to obtain a comparison result;

[0028] S103: When the comparison result indicates that the time stamp interval between the first image in the first image sequence and the second image in the second image sequence is less than or equal to a preset threshold, determine the first image and the second image as a target image group for alignment.

[0029] The method for determining the target image group of the embodiment of the present application can obtain the first image sequence of the first camera and the timestamp of each image in the first image sequence, and obtain the second image sequence of the second camera and the timestamp of each image in the second image sequence, the first camera and the second camera are cameras set at different shooting angles of the vehicle; compare the timestamps of the images in the first image sequence with the timestamps of the images in the second image sequence to obtain a comparison result; finally, when the comparison result indicates that the timestamp interval between the first image in the first image sequence and the second image in the second image sequence is less than or equal to a preset threshold, the first image and the second image are determined as the aligned target image group. In this way, the embodiment determines the first image and the second image whose timestamp interval is less than or equal to the preset threshold as the aligned target image group by comparing the timestamp interval between the timestamp of the first image in the first image sequence and the timestamp of the second image in the second image sequence, so as to realize the alignment of images collected by cameras at different shooting angles in the vehicle, thereby improving the accuracy of the vehicle intelligent interaction model in identifying interaction information.

[0030] The specific implementation methods of the above steps are introduced below.

[0031] In S101, the first camera and the second camera are cameras set at different shooting angles of the vehicle. There may be at least two cameras set at different shooting angles on the vehicle, and the first camera and the second camera may be any two of them.

[0032] The first image sequence is a sequence of at least one frame of images inside the vehicle captured by the first camera, and the second image sequence is a sequence of at least one frame of images inside the vehicle captured by the second camera.

[0033] The above-mentioned obtaining of the first image sequence of the first camera and the timestamp of each image in the first image sequence may, exemplarily, be that when the first camera captures an image, the image captured by the first camera and the timestamp of the image are stored in a cache queue associated with the first camera, wherein the cache queues associated with the first camera and the second camera are different; at least one frame of image and the timestamp of each image are read from the cache queue associated with the first camera, and the first image sequence includes at least one frame of image.

[0034] The above-mentioned method of obtaining the second image sequence of the second camera and the timestamp of each image in the second image sequence is similar to the method of obtaining the first image sequence of the first camera and the timestamp of each image in the first image sequence, and will not be repeated here.

[0035] In S102, the timestamps of the images in the first image sequence are compared with the timestamps of the images in the second image sequence to obtain a comparison result. For example, each image in the first image sequence is paired with each image in the second image sequence to obtain at least one paired image group, each paired image group includes a frame image in the first image sequence and a frame image in the second image sequence; the timestamp interval between two frames of images in each paired image group in at least one paired image group is calculated, wherein the comparison result may include the timestamp interval corresponding to at least one paired image group. Alternatively, the timestamps of the k-th frame image in the first target image sequence are compared with the images in the second target image sequence in sequence, wherein the comparison result may include the timestamp interval between the k-th frame image and each image in the second target image sequence, wherein the first target image sequence is one of the first image sequence and the second image sequence, and the second target image sequence is the other of the first image sequence and the second image sequence.

[0036] In S103, the preset threshold, illustratively, may be 33ms, or may be a frame rate set according to user requirements, thereby determining a threshold for timestamp interval comparison. For example, if the frame rate is 30FPS, i.e., 1 frame is 33ms, the corresponding preset threshold is 33ms. In this embodiment, the preset threshold is not limited to 33ms, but may be other values, which are not specifically limited here.

[0037] The first image is an image in a first image sequence, and the second image is an image in a second image sequence.

[0038] In the above, when the comparison result indicates that the time stamp interval between the first image in the first image sequence and the second image in the second image sequence is less than or equal to the preset threshold, the first image and the second image are determined as the target image group for alignment. Exemplarily, when the time stamp interval corresponding to the first image group in at least one pair of image groups is less than or equal to the preset threshold, the two frames of images in the first image group are determined as the target image group for alignment. Alternatively, when the comparison result indicates that the time stamp interval between the k-th frame image and the third image in the second target image sequence is less than or equal to the preset threshold, the k-th frame image and the third image are determined as the second image group, the comparison of the k-th frame image with the images in the second target image sequence is stopped, the third image in the second target image sequence and the images before the third image are eliminated, the next frame image of the k-th frame image is updated to the k-th frame image, and the time stamp comparison of the k-th frame image in the first target image sequence with the images in the second target image sequence is re-executed, wherein the target image group for alignment includes the second image group.

[0039] In some embodiments, the above S102 may specifically include:

[0040] Pairing the images respectively included in the first image sequence and the second image sequence in pairs to obtain at least one paired image group, each paired image group including one frame of image in the first image sequence and one frame of image in the second image sequence;

[0041] Calculating the time stamp interval between two frames of images in each paired image group, wherein the comparison result includes the time stamp interval corresponding to at least one set of paired image groups;

[0042] The above S103 may specifically include:

[0043] When the time stamp interval corresponding to the first image group in at least one paired image group is less than or equal to a preset threshold, two frames of images in the first image group are determined as the target image group for alignment.

[0044] The pairing of the images in the first image sequence with the images in the second image sequence may be performed in pairs according to the order of the images in the first image sequence and the order of the images in the second image sequence.

[0045] Each of the paired image groups includes a frame of image in the first image sequence and a frame of image in the second image sequence.

[0046] The above comparison result may include a timestamp interval corresponding to at least one set of paired image groups.

[0047] In this embodiment, by pairing each image in the first image sequence with each image in the second image sequence, at least one set of paired image groups is obtained. When the timestamp interval corresponding to the first image group in at least one set of paired image groups is less than or equal to a preset threshold, the two frames of images in the first image group can be determined as the aligned target image group, thereby achieving alignment of images captured by cameras with different shooting angles in the vehicle.

[0048] In some embodiments, the above S102 may specifically include:

[0049] Comparing the timestamps of the k-th frame image in the first target image sequence with the images in the second target image sequence in sequence, wherein the comparison result includes the timestamp intervals between the k-th frame image and each image in the second target image sequence, the first target image sequence is one of the first image sequence and the second image sequence, and the second target image sequence is the other of the first image sequence and the second image sequence;

[0050] The above S103 may specifically include:

[0051] When the comparison result indicates that the timestamp interval between the k-th frame image and the third image in the second target image sequence is less than or equal to a preset threshold, the k-th frame image and the third image are determined as the second image group, the comparison of the k-th frame image with the images in the second target image sequence is stopped, the third image in the second target image sequence and the image before the third image are eliminated, the next frame image of the k-th frame image is determined as the k-th frame image, and the timestamp comparison of the k-th frame image in the first target image sequence with the images in the second target image sequence is re-executed, wherein the aligned target image group includes the second image group.

[0052] The first target image sequence is one of the first image sequence and the second image sequence, and the second target image sequence is the other of the first image sequence and the second image sequence.

[0053] The above comparison result may also include the time stamp interval between the k-th frame image and each image in the second target image sequence.

[0054] The third image is an image whose timestamp interval with the k-th frame image is less than or equal to a preset threshold.

[0055] The above-mentioned aligned target image group may further include a second image group, wherein the second image group includes the k-th frame image and the third image whose timestamp interval is less than or equal to a preset threshold.

[0056] In this embodiment, by comparing the timestamps of the kth frame image in the first target image sequence with the images in the second target image sequence in sequence, when the comparison result indicates that the timestamp interval between the kth frame image and the third image in the second target image sequence is less than or equal to a preset threshold, the kth frame image and the third image are determined as the second image group, and the aligned target image group includes the second image group, which can improve the accuracy of the confirmation of the second image group, thereby achieving accurate alignment of images captured by cameras with different shooting angles in the vehicle.

[0057] In some embodiments, if there is no third image in the second target image sequence whose timestamp interval with the k-th frame image is less than or equal to a preset threshold, the next frame image of the k-th frame image is updated to the k-th frame image, and the timestamp comparison of the k-th frame image in the first target image sequence with the images in the second target image sequence is re-executed.

[0058] In this embodiment, when there is no third image in the second target image sequence whose timestamp interval with the k-th frame image is less than or equal to a preset threshold, the next frame image of the k-th frame image can be updated to the k-th frame image, and the timestamp comparison of the k-th frame image in the first target image sequence with the images in the second target image sequence is re-executed, thereby improving the efficiency of confirming the second image group, and further improving the alignment efficiency of images captured by cameras with different shooting angles in the vehicle.

[0059] As an implementation of the present application, in order to avoid data reading errors, before the above S101, the following may also be included:

[0060] When the first camera captures an image, the image captured by the first camera and the timestamp of the image are stored in a cache queue associated with the first camera, wherein the cache queues associated with the first camera and the second camera are different;

[0061] The above S101 may specifically include:

[0062] At least one frame of image and a timestamp of each image are read from a cache queue associated with the first camera, wherein the first image sequence includes at least one frame of image.

[0063] In this embodiment, the cache queues associated with the first camera and the second camera are different. When the first camera captures an image, the image captured by the first camera and the timestamp of the image are stored in the cache queue associated with the first camera. Therefore, when obtaining the first image sequence of the first camera and the timestamp of each image in the first image sequence, they can be directly read in the cache queue associated with the first camera, thereby avoiding data reading errors.

[0064] Similarly, in some embodiments, before the above S101, the following steps may also be included:

[0065] When the second camera captures an image, the image captured by the second camera and the timestamp of the image are stored in a cache queue associated with the second camera, wherein the cache queue associated with the second camera is different from the cache queue associated with the first camera;

[0066] The above S101 may specifically include:

[0067] At least two frames of images and a timestamp of each image are read from a cache queue associated with the second camera, where the second image sequence includes at least two frames of images.

[0068] In some embodiments, the cache queue associated with the first camera includes the i-1th frame image and the timestamp of the i-1th frame image;

[0069] In the case where the first camera captures an image, storing the image captured by the first camera and the timestamp of the image in a cache queue associated with the first camera may specifically include:

[0070] When the first camera captures the i-th frame image, the i-th frame image is compared with the i-1-th frame image to obtain a comparison result, where the comparison result is used to indicate whether a preset area in the i-th frame image changes compared with the i-1-th frame image;

[0071] When the comparison result indicates that a preset area in the i-th frame image changes compared with the i-1-th frame image, the i-th frame image and the timestamp of the i-th frame image are stored in a cache queue associated with the first camera.

[0072] The cache queue associated with the first camera may include the i-1th frame image and the timestamp of the i-1th frame image.

[0073] The above comparison result can be used to indicate whether the preset area in the i-th frame image has changed compared with the i-1-th frame image. The preset area can be, for example, an image of the user's hand area or an image of the seat area in the car. This embodiment is not limited thereto and can also be set according to user needs, which is not specifically limited here.

[0074] In this embodiment, the comparison result indicates that there is a change in the preset area in the i-th frame image compared to the i-1-th frame image, which means that the user in the car may interact with the vehicle computer through gestures or seat changes. Therefore, the i-th frame image and the timestamp of the i-th frame image are stored in the cache queue associated with the first camera for subsequent image alignment and image recognition processing.

[0075] In some embodiments, when the first camera captures an image, storing the image captured by the first camera and the timestamp of the image in a cache queue associated with the first camera may further include:

[0076] When the comparison result indicates that the preset area in the i-th frame image has not changed compared with the i-1-th frame image, the i-th frame image and the timestamp of the i-th frame image replace the i-1-th frame image and the timestamp of the i-1-th frame image in the cache queue associated with the first camera.

[0077] In this embodiment, when the comparison result indicates that the preset area in the i-th frame image has not changed compared with the i-1 frame image, the i-th frame image and the timestamp of the i-th frame image are replaced with the i-1 frame image and the timestamp of the i-1 frame image in the cache queue associated with the first camera, thereby saving storage space in the cache queue.

[0078] In order to facilitate the understanding of the method for determining the target image group in the embodiment of the present application, the application process of the method for determining the target image group is described here, as follows:

[0079] The vehicle computer receives images captured by at least two cameras in different directions in the vehicle cabin. After receiving the image, the vehicle computer first aligns the image (equivalent to the above-mentioned i-th frame image) with the image previously generated by the camera (equivalent to the above-mentioned i-1-th frame image) according to the different cameras. This process is performed separately for each camera (such as camera 1). The vehicle computer stores the image generated by camera 1 and compares this image with the image generated previously. Then, it completes the replacement operation in combination with the judgment conditions, and uses the time when the new image is generated as the time when the stored image is generated; after that, the vehicle computer completes the image alignment work, that is, aligns the time when the images captured by different cameras are generated, and inputs the time-aligned images into the recognition model for passenger position or gesture recognition, and the recognition model outputs the corresponding recognition results.

[0080] Among them, the replacement judgment process is: for example, for a stored image and a newly generated image, pixels in specific areas of the two images are judged. When there is a change or the change ratio is greater than a set value, the stored image is replaced by the newly generated image. The specific area can be the area corresponding to each seat pre-set by the user, etc.

[0081] The alignment process is as follows: first, store the IR (equivalent to the first camera) and TOF (equivalent to the second camera) camera data in the queue respectively, and compare the timestamps of the IR and TOF camera data; if the IR timestamp of two adjacent frames of IR and TOF timestamps is less than 33ms of the TOF timestamp, this frame of IR data will be discarded; if the IR timestamp is greater than 33ms of the TOF timestamp, the earlier TOF data will be discarded; finally, two adjacent frames of IR and TOF data with a time interval less than 33ms are simultaneously transmitted to the visual algorithm model (i.e., a type of recognition model) to detect the occupancy information of personnel in the cabin.

[0082] The time-aligned images are input into the recognition model, and the recognition model outputs the corresponding recognition results. For example, the deep learning model can be trained at each position to output the corresponding scene conditions. Because the camera position is fixed, the imaging position on the screen is basically fixed. Traditionally, the image is divided into 4 areas to detect whether there is a person in the corresponding image position coordinate range.

[0083] In this embodiment, images captured by cameras in different orientations are aligned by time stamps, thereby improving the accuracy of interactive information recognition, thereby making the recognition result generated by the recognition model more accurate.

[0084] Based on the method for determining the target image group provided in the above embodiment, the present application also provides a specific implementation of a device for determining the target image group. Please refer to the following embodiment.

[0085] like Figure 2 As shown, the device 200 for determining a target image group provided in an embodiment of the present application may include the following modules: an acquisition module 201 , a comparison module 202 and a determination module 203 .

[0086] An acquisition module 201 is used to acquire a first image sequence from a first camera and a timestamp of each image in the first image sequence, and to acquire a second image sequence from a second camera and a timestamp of each image in the second image sequence, wherein the first camera and the second camera are cameras at different shooting angles of the vehicle;

[0087] A comparison module 202, configured to compare the time stamps of the images in the first image sequence with the time stamps of the images in the second image sequence to obtain a comparison result;

[0088] The determination module 203 is configured to determine the first image and the second image as a target image group for alignment when the comparison result indicates that the timestamp interval between the first image in the first image sequence and the second image in the second image sequence is less than or equal to a preset threshold.

[0089] The device for determining the target image group of the embodiment of the present application can obtain the first image sequence of the first camera and the timestamp of each image in the first image sequence, and obtain the second image sequence of the second camera and the timestamp of each image in the second image sequence, wherein the first camera and the second camera are cameras at different shooting angles of the vehicle; compare the timestamps of the images in the first image sequence with the timestamps of the images in the second image sequence to obtain a comparison result; finally, when the comparison result indicates that the timestamp interval between the first image in the first image sequence and the second image in the second image sequence is less than or equal to a preset threshold, the first image and the second image are determined as the aligned target image group. In this way, the embodiment determines the first image and the second image whose timestamp interval is less than or equal to the preset threshold as the aligned target image group by comparing the timestamp interval between the timestamp of the first image in the first image sequence and the timestamp of the second image in the second image sequence, so as to realize the alignment of images collected by cameras at different shooting angles in the vehicle, thereby improving the accuracy of the vehicle intelligent interaction model in identifying interaction information.

[0090] In some embodiments, the comparison module 202 may specifically include:

[0091] a pairing unit, configured to pair the images respectively included in the first image sequence and the second image sequence in pairs to obtain at least one set of paired image groups, each of which includes one frame of image in the first image sequence and one frame of image in the second image sequence;

[0092] A calculation unit, used to calculate the time stamp interval between two frames of images in each paired image group, wherein the comparison result includes the time stamp interval corresponding to at least one set of paired image groups;

[0093] The above-mentioned determination module 203 may specifically include:

[0094] The first determining unit is configured to determine two frames of images in the first image group as a target image group for alignment when a timestamp interval corresponding to the first image group in at least one paired image group is less than or equal to a preset threshold.

[0095] In some embodiments, the comparison module 202 may specifically include:

[0096] a comparing unit, configured to compare the timestamps of the k-th frame image in the first target image sequence with the images in the second target image sequence in sequence, wherein the comparison result includes the timestamp interval between the k-th frame image and each image in the second target image sequence, the first target image sequence is one of the first image sequence and the second image sequence, and the second target image sequence is the other of the first image sequence and the second image sequence;

[0097] The above-mentioned determination module 203 may specifically include:

[0098] A second determination unit is used to determine the k-th frame image and the third image as the second image group, stop comparing the k-th frame image with the images in the second target image sequence, eliminate the third image and the image before the third image in the second target image sequence, determine the next frame image of the k-th frame image as the k-th frame image, and re-execute the timestamp comparison of the k-th frame image in the first target image sequence with the images in the second target image sequence in sequence, wherein the aligned target image group includes the second image group.

[0099] As an implementation of the present application, in order to avoid data reading errors, the above-mentioned device 200 may further include:

[0100] A storage unit, configured to store the image captured by the first camera and the timestamp of the image in a cache queue associated with the first camera when the first camera captures the image, wherein the cache queues associated with the first camera and the second camera are different;

[0101] The acquisition module 201 may specifically include:

[0102] The acquisition unit is used to read at least one frame of image and the timestamp of each image from a cache queue associated with the first camera, and the first image sequence includes at least one frame of image.

[0103] In some embodiments, the cache queue associated with the first camera includes the i-1th frame image and the timestamp of the i-1th frame image;

[0104] The above storage unit may specifically include:

[0105] A comparison subunit is used to compare the i-th frame image with the i-1-th frame image when the first camera captures the i-th frame image, and obtain a comparison result, wherein the comparison result is used to indicate whether a preset area in the i-th frame image changes compared with the i-1-th frame image;

[0106] The storage subunit is used to store the i-th frame image and the timestamp of the i-th frame image in a cache queue associated with the first camera when the comparison result indicates that the preset area in the i-th frame image changes compared with the i-1-th frame image.

[0107] In some embodiments, the storage unit may further include:

[0108] The replacement subunit is used to replace the i-th frame image and the timestamp of the i-th frame image with the i-1 frame image and the timestamp of the i-1 frame image in the cache queue associated with the first camera when the comparison result indicates that the preset area in the i-th frame image has not changed compared with the i-1 frame image.

[0109] Based on the method for determining the target image group provided in the above embodiment, the present application also provides a specific implementation of the electronic device. Please refer to the following embodiment.

[0110] Figure 3 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application is shown.

[0111] The electronic device may include a processor 301 and a memory 302 storing computer program instructions.

[0112] Specifically, the processor 301 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0113] The memory 302 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 302 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 302 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 302 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 302 is a non-volatile solid-state memory.

[0114] In certain embodiments, the memory 302 may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Thus, in general, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0115] The processor 301 reads and executes computer program instructions stored in the memory 302 to implement any one of the methods for determining the target image group in the above embodiments.

[0116] In one example, the electronic device may further include a communication interface 303 and a bus 310. Figure 3 As shown, the processor 301, the memory 302, and the communication interface 303 are connected via a bus 310 and communicate with each other.

[0117] The communication interface 303 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0118] Bus 310 includes hardware, software or both, and the parts of electronic equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 310 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the application considers any suitable bus or interconnection.

[0119] The electronic device can execute the method for determining the target image group in the embodiment of the present application, thereby realizing the combination Figure 1 and Figure 2 A method and device for determining a target image group are described.

[0120] In addition, in combination with the method for determining the target image group in the above embodiment, the embodiment of the present application can provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the methods for determining the target image group in the above embodiment is implemented.

[0121] In combination with the method for determining a target image group in the above embodiment, the present application embodiment may provide a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes any of the above methods for determining a target image group.

[0122] In combination with the method for determining the target image group in the above embodiment, the embodiment of the present application can provide a vehicle for implementation. The vehicle includes: an electronic device for implementing any of the above methods for determining the target image group.

[0123] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0124] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0125] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0126] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0127] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

[0128] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprises" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A method for determining a target image group, It is characterized in that include: Acquire a first image sequence from a first camera and a timestamp of each image in the first image sequence, and acquire a second image sequence from a second camera and a timestamp of each image in the second image sequence, wherein the first camera and the second camera are cameras set at different shooting angles of the vehicle; comparing the time stamps of the images in the first image sequence with the time stamps of the images in the second image sequence to obtain a comparison result; When the comparison result indicates that the time stamp interval between a first image in the first image sequence and a second image in the second image sequence is less than or equal to a preset threshold, the first image and the second image are determined as an aligned target image group.

2. The method according to claim 1, It is characterized in that The comparing the time stamps of the images in the first image sequence with the time stamps of the images in the second image sequence to obtain a comparison result includes: Pairing the images respectively included in the first image sequence and the second image sequence in pairs to obtain at least one set of paired image groups, each of the paired image groups including one frame of image in the first image sequence and one frame of image in the second image sequence; Calculating the time stamp interval between two frames of images in each of the paired image groups, wherein the comparison result includes the time stamp interval corresponding to the at least one paired image group; The step of determining the first image and the second image as a target image group for alignment when the comparison result indicates that a timestamp interval between a first image in the first image sequence and a second image in the second image sequence is less than or equal to a preset threshold value comprises: When the time stamp interval corresponding to the first image group in the at least one paired image group is less than or equal to a preset threshold, two frames of images in the first image group are determined as a target image group for alignment.

3. The method according to claim 1, It is characterized in that The comparing the time stamps of the images in the first image sequence with the time stamps of the images in the second image sequence to obtain a comparison result includes: Comparing the k-th frame image in the first target image sequence with the images in the second target image sequence in sequence by time stamp, wherein the comparison result includes the time stamp interval between the k-th frame image and each image in the second target image sequence, the first target image sequence is one of the first image sequence and the second image sequence, and the second target image sequence is the other of the first image sequence and the second image sequence; The step of determining the first image and the second image as a target image group for alignment when the comparison result indicates that a timestamp interval between a first image in the first image sequence and a second image in the second image sequence is less than or equal to a preset threshold value comprises: When the comparison result indicates that the timestamp interval between the k-th frame image and the third image in the second target image sequence is less than or equal to a preset threshold, the k-th frame image and the third image are determined as a second image group, the comparison of the k-th frame image with the images in the second target image sequence is stopped, the third image and the image before the third image in the second target image sequence are eliminated, the next frame image of the k-th frame image is determined as the k-th frame image, and the timestamp comparison of the k-th frame image in the first target image sequence with the images in the second target image sequence is re-executed, wherein the aligned target image group includes the second image group.

4. The method according to claim 1, It is characterized in that Before acquiring the first image sequence of the first camera and the timestamp of each image in the first image sequence, the method further includes: When the first camera captures an image, storing the image captured by the first camera and a timestamp of the image in a cache queue associated with the first camera, wherein the cache queues associated with the first camera and the second camera are different; The acquiring a first image sequence of a first camera and a timestamp of each image in the first image sequence includes: At least one frame of image and a timestamp of each image are read from a cache queue associated with the first camera, wherein the first image sequence includes the at least one frame of image.

5. The method according to claim 4, It is characterized in that The cache queue associated with the first camera includes an i-1th frame image and a timestamp of the i-1th frame image; When the first camera captures an image, storing the image captured by the first camera and the timestamp of the image in a cache queue associated with the first camera includes: When the first camera captures the i-th frame image, compare the i-th frame image with the i-1-th frame image to obtain a comparison result, where the comparison result is used to indicate whether a preset area in the i-th frame image changes compared with the i-1-th frame image; When the comparison result indicates that a preset area in the i-th frame image changes compared with the i-1-th frame image, the i-th frame image and the timestamp of the i-th frame image are stored in a cache queue associated with the first camera.

6. The method according to claim 5, It is characterized in that Also includes: When the comparison result indicates that a preset area in the i-th frame image has not changed compared with the i-1-th frame image, the i-th frame image and the timestamp of the i-th frame image are replaced with the i-1-th frame image and the timestamp of the i-1-th frame image in the cache queue associated with the first camera.

7. A device for determining a target image group, It is characterized in that include: an acquisition module, configured to acquire a first image sequence from a first camera and a timestamp of each image in the first image sequence, and to acquire a second image sequence from a second camera and a timestamp of each image in the second image sequence, wherein the first camera and the second camera are cameras set at different shooting angles of the vehicle; a comparison module, configured to compare the time stamps of the images in the first image sequence with the time stamps of the images in the second image sequence to obtain a comparison result; A determination module is used to determine the first image and the second image as an aligned target image group if the comparison result indicates that the timestamp interval between the first image in the first image sequence and the second image in the second image sequence is less than or equal to a preset threshold.

8. An electronic device, It is characterized in that The electronic device comprises: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the method for determining the target image group according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the method for determining a target image group according to any one of claims 1 to 6 is implemented.

10. A vehicle, It is characterized in that include: An electronic device, wherein the electronic device is used to implement the method for determining a target image group as described in any one of claims 1 to 6.

Images