An image stitching method, device and electronic equipment

Abstract

This application provides an image stitching method, apparatus, and electronic device, comprising: obtaining at least one video frame captured for a target area; identifying an entering video frame, an associated video frame, and an exit video frame from the at least one video frame; wherein the entering video frame, the associated video frame, and the exit video frame contain the same moving object; the entering video frame is an image captured when the moving object enters the target area, the associated video frame is an image captured after the moving object enters the target area and before it exits the target area, and the exit video frame is an image captured when the moving object exits the target area; and stitching the entering video frame, the associated video frame, and the exit video frame together. Using the method provided in this application, multiple video frames to be stitched can be accurately identified from video captured by a camera, from the moving object's entry into the target area to its exit from the target area.

Description

Technical Field

[0001] This application relates to the field of image processing, and more particularly to an image stitching method, apparatus, and electronic device. Background Technology

[0002] There is a need to photograph moving objects passing through a target area in many scenarios. For example, cameras are set up at park entrances and exits, highway toll booths, and toll gates to capture images of moving objects and obtain relevant information about the moving objects by recognizing the images, in order to facilitate the photographing of the moving objects.

[0003] However, when the moving object is too large, the cameras deployed in the target area can often only capture a local image of the moving object when it passes through the target area. Therefore, it is necessary to stitch together the local images of multiple moving objects to obtain a global image of the moving object.

[0004] In existing methods, sensors are typically placed at the entrance and exit of the target area. The entrance sensor generates an entry time after detecting a moving object, and the exit sensor generates an exit time after detecting a moving object. The electronic device can select video frames captured between the entry and exit times from the video frames captured by the camera and use these video frames as the video frames to be stitched together.

[0005] The downside of this approach is:

[0006] On the one hand, the need to deploy additional inlet and outlet sensors increases hardware deployment costs.

[0007] On the other hand, this method determines the video frames from when a moving object enters to when it leaves the target area based on time. This may result in the determination of video frames that are not local images of the moving object (such as video frames that only contain the background of the target area), causing the determined images to be stitched to be inaccurate. Summary of the Invention

[0008] In view of this, this application provides an image stitching method, apparatus, and electronic device for accurately identifying multiple video frames to be stitched from a moving object entering a target area to leaving the target area in a video captured by a camera.

[0009] Specifically, this application is implemented through the following technical solution:

[0010] According to a first aspect of this application, an image stitching method is provided, the method comprising:

[0011] Obtain at least one video frame captured for a target area; the at least one video frame includes at least one target video frame;

[0012] Image recognition is performed on the target video frame to obtain an entry video frame, an associated video frame, and an exit video frame; wherein the same moving object exists in the entry video frame, the associated video frame, and the exit video frame; the entry video frame is an image captured when the moving object enters the target area, the associated video frame is an image captured when the moving object enters the target area and before it exits the target area, and the exit video frame is an image captured when the moving object exits the target area;

[0013] The incoming video frame, the associated video frame, and the outgoing video frame are spliced ​​together.

[0014] According to a second aspect of this application, an image stitching apparatus is provided, the apparatus comprising:

[0015] Acquisition unit, used to acquire at least one video frame captured for the target area;

[0016] The identification unit is configured to identify an entering video frame, an associated video frame, and an exit video frame from the at least one video frame; wherein the entering video frame, the associated video frame, and the exit video frame contain the same moving object; the entering video frame is an image captured when the moving object enters the target area, the associated video frame is an image captured after the moving object enters the target area and before it exits the target area, and the exit video frame is an image captured when the moving object exits the target area;

[0017] The splicing unit is used to splice the incoming video frame, the associated video frame, and the outgoing video frame.

[0018] According to a third aspect of this application, an electronic device is provided, the electronic device including a readable storage medium and a processor;

[0019] The readable storage medium is used to store machine-executable instructions;

[0020] The processor is configured to read the machine-executable instructions on the readable storage medium and execute the instructions to implement the image stitching method.

[0021] According to a fourth aspect of this application, a computer-readable storage medium is provided, wherein a computer program is stored therein, and the computer program, when executed by a processor, implements the above-described image stitching method.

[0022] According to a fifth aspect of this application, a computer program is provided, which is stored in a computer-readable storage medium, and when a processor executes the computer program, causes the processor to implement the above-described image stitching method.

[0023] This application uses image recognition technology to identify, from at least one video frame of the target area, the video frame entering the target area, the associated video frame and the video frame exiting the target area, thereby identifying multiple video frames to be stitched together from the entry of the moving object into the target area to the exit of the target area.

[0024] On the one hand, this method does not require the configuration of additional inlet and outlet sensors, so it can greatly reduce hardware deployment costs;

[0025] On the other hand, since this application does not use video frames acquired between the entry and exit times as video frames to be stitched, but rather uses image recognition technology to identify video frames to obtain video frames to be stitched, this application can accurately identify these video frames to be stitched that contain local images of moving objects, and can effectively prevent video frames containing local images of non-moving objects from being mixed in with the video frames to be stitched. Attached Figure Description

[0026] Figure 1 This is a flowchart illustrating an exemplary embodiment of the present application of an image stitching method;

[0027] Figure 2 This is a schematic diagram illustrating an image stitching method according to an exemplary embodiment of this application;

[0028] Figure 3 This is a schematic diagram illustrating the first condition in an exemplary embodiment of this application;

[0029] Figure 4 This is a schematic diagram illustrating the third condition in an exemplary embodiment of this application;

[0030] Figure 5 This is a flowchart illustrating another splicing method in an exemplary embodiment of this application;

[0031] Figure 6 This is a hardware structure diagram of an electronic device illustrated in an exemplary embodiment of this application;

[0032] Figure 7 This is a block diagram of an image stitching device according to an exemplary embodiment of this application. Detailed Implementation

[0033] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0034] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0035] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0036] In the traditional approach, an entrance sensor is installed at the entrance of the target area, and an exit sensor is installed at the exit. When the entrance sensor detects a moving object covering the entrance line of the target area, it records the current time as the object's entry time. When the exit sensor detects a moving object covering the exit line of the target area, it records the current time as the object's exit time. The electronic device can then, based on these entry and exit times, find multiple video frames captured between the entry and exit times from the video footage of the target area captured by cameras positioned within the target area, and stitch these multiple video frames together to obtain a global image of the moving object.

[0037] However, the downside of this splicing method is:

[0038] On the one hand, this traditional approach requires additional deployment of inlet and outlet sensors, increasing hardware deployment costs.

[0039] On the other hand, this method determines the video frames from when the moving object enters to when it leaves the target area based on time. This may result in the determination of video frames that are not local images of the moving object (such as video frames that only contain the background of the target area), causing inaccurate identification of the image to be stitched.

[0040] For example, suppose a moving object stops moving forward for a period of time in front of the entrance line, and during this period of stopping, the moving object does not enter the field of view of the camera positioned in the target area.

[0041] Since the moving object has reached the entry line, the sensor records the current moment as the entry time. However, after recording the entry time, the moving object stops moving for a period of time. During this period of cessation, the moving object does not enter the field of view of the camera positioned in the target area. At this time, using the above method will stitch together a large number of images reflecting the target area, resulting in a large amount of background of the target area in the stitched global image.

[0042] This application provides an image stitching method. This application uses image recognition technology to identify, from at least one video frame acquired for a target area, an entry video frame, an associated video frame and an exit video frame, thereby identifying multiple video frames to be stitched from the entry to the exit of a moving object from the target area.

[0043] As can be seen from the above description, on the one hand, compared with acquiring the video frames to be stitched through sensors, the method of this application does not require the configuration of additional inlet and outlet sensors, so the method of this application can greatly reduce the hardware deployment cost;

[0044] On the other hand, since this application does not use video frames acquired between the entry and exit times as video frames to be stitched, but rather uses image recognition technology to identify video frames to obtain video frames to be stitched, this application can accurately identify these video frames to be stitched that contain local images of moving objects, and can effectively prevent video frames containing local images of non-moving objects from being mixed in with the video frames to be stitched.

[0045] Taking the above example again, suppose that when the moving object stops moving forward for a period of time in front of the entrance line, during this period of stopping, the moving object does not enter the field of view of the camera positioned in the target area.

[0046] Using the above method of acquiring images to be stitched through sensors will result in a large number of background images being used as the images to be stitched, ultimately leading to the stitched global image containing a large number of background images.

[0047] Since this application directly identifies the entering video frame based on image recognition, even if the camera captures an image containing the background of the target area due to the moving object stopping, this application will not use the captured background image as the entering video frame, but will use the video frame captured when the moving object enters the target area as the entering video frame, thereby avoiding the presence of a large number of background images in the stitched global image.

[0048] The image stitching method provided in this application is described in detail below.

[0049] It should be noted that the moving object mentioned in this application refers to an object that can move, such as a vehicle, a ship or other means of transportation, or a movable cargo box or goods. This is only an illustrative example of the moving object and is not specifically limited to it.

[0050] The target area mentioned in this application refers to a pre-defined area. For example, in a highway toll payment scenario, the target area could be the entrance or exit area of ​​the highway. Similarly, in a parking lot scene where vehicles are parked and paying tolls, the target area could be the entrance or exit area of ​​the parking lot. This is merely an illustrative example of the target area and is not intended to impose any specific limitations.

[0051] Furthermore, this application does not limit the scenarios in which the image stitching method proposed in this application is applicable. The image stitching method of this application can be used in any scenario where the camera cannot capture a global view of a moving object.

[0052] See Figure 1 , Figure 1 This is a flowchart illustrating an exemplary embodiment of an image stitching method. The method can be applied to an electronic device, such as a camera or a server. This is merely an example of an electronic device and is not specifically limited thereto.

[0053] Step 101: The electronic device acquires at least one video frame captured for the target area.

[0054] 1) Describe how at least one video frame was captured.

[0055] In one alternative implementation, a regional camera is deployed inside the target area. An external camera is deployed outside the entrance to the target area. When the external camera detects a moving object, it can send a capture signal to the regional camera, which, upon receiving the capture signal, captures at least one video frame of the target area.

[0056] For example, in a highway tolling scenario, a front-end camera is installed outside the highway entrance area, and a vehicle body camera is installed inside the area. When the front-end camera detects a vehicle, it sends a acquisition signal to the vehicle body camera, which then acquires at least one video frame for the target area.

[0057] 2) Describe how the electronic device acquires at least one video frame for the target area.

[0058] In one alternative implementation, when the electronic device is a server, the server can acquire the at least one video frame from the area camera, or the area camera can actively report at least one video frame to the server.

[0059] When the electronic device is a regional camera, the regional camera can directly capture at least one video frame of the target area.

[0060] It should be noted that the above-mentioned at least one video frame can be a temporally consecutive video frame, without specific limitations here.

[0061] Step 102: The electronic device identifies an entering video frame, an associated video frame, and an exit video frame from the at least one video frame; wherein the entering video frame, the associated video frame, and the exit video frame contain the same moving object; the entering video frame is an image captured when the moving object enters the target area, the associated video frame is an image captured after the moving object enters the target area and before it exits the target area, and the exit video frame is an image captured when the moving object exits the target area.

[0062] In implementation, electronic devices can use image recognition technology, combined with preset entry and exit conditions, to identify an entry video frame, associated video frames, and exit video frames from at least one video frame.

[0063] For example, taking a vehicle as the moving object, such as Figure 2 As shown, step 102 is to identify, from at least one video frame, such as Figure 2 Figure a shows the incoming video frame, the associated video frame, and the outgoing video frame, for example... Figure 2 In Figure a, the first image in the first row is the incoming video frame, the third image in the third row is the outgoing video frame, and the other images are related video frames.

[0064] The purpose of this application is to identify Figure 2 The inbound video frame, the associated video frame, and the outbound video frame shown in Figure (a) are then stitched together to form... Figure 2 Figure (b) shows a global view of the vehicle.

[0065] In one alternative implementation, the electronic device can determine, from at least one video frame, an incoming video frame that meets preset incoming conditions, and an outgoing video frame that meets outgoing conditions. The electronic device can then use video frames between the incoming and outgoing video frames as associated video frames. For example, the electronic device can use video frames captured after the incoming video frame and before the outgoing video frame from the at least one video frame as associated video frames.

[0066] Of course, in practical applications, electronic devices can also identify incoming video frames, outgoing video frames, and associated video frames in other ways. This is just an example and is not specifically limited.

[0067] The steps 1021 to 1023 below describe in detail the process of "the electronic device determining, from at least one video frame, an entry video frame that meets preset entry conditions, and an exit video frame that meets exit conditions from at least one video frame. Then, the electronic device can use the video frame between the entry video frame and the exit video frame as a associated video frame".

[0068] Step 1021: For each video frame in the at least one video frame, detect whether an incoming video frame associated with this video frame and acquired earlier than this video frame has been identified.

[0069] It should be noted that this article uses one frame from the target video frame as an example for description, so for ease of description, this frame will be referred to as this video frame.

[0070] In one optional detection method, the electronic device sets a variable with an initial value of 0. A value of 0 indicates that no related inbound video frame has been identified that was captured earlier than the current video frame. When the electronic device identifies a video frame that meets the inbound condition, it changes the value of the variable to 1. A value of 1 indicates that a related inbound video frame that was captured earlier than the current video frame has been identified.

[0071] During detection, the electronic device can detect the value of this variable. If the value of this variable is 0, it indicates that an incoming video frame associated with this video frame and acquired earlier than this video frame has not yet been identified. If the value of this variable is 1, it indicates that an incoming video frame associated with this video frame and acquired earlier than this video frame has been identified.

[0072] Of course, in practical applications, electronic devices can also use other methods to achieve the detection of "whether an inbound video frame associated with this video frame and acquired earlier than this video frame has been identified". For example, when the electronic device identifies each video frame, it generates an identification result record. During detection, the electronic device can detect whether the identified inbound video frame has been recorded in the identification result record. This is only an illustrative example of the detection method in step 1021, and it is not specifically limited thereto.

[0073] In this application example, if an incoming video frame associated with this video frame and acquired earlier than this video frame has been identified, then step 1022 is executed. If no incoming video frame associated with this video frame and acquired earlier than this video frame has been identified, then step 1023 is executed.

[0074] Step 1022: If yes, perform image recognition on the video frame. If the recognition result does not meet the preset exit conditions, then determine that the video frame is an associated video frame related to the inbound video frame; if the image recognition result of the video frame meets the exit conditions, then determine that the video frame is an outbound video frame related to the inbound video frame.

[0075] In this embodiment of the application, since the video frames are acquired in chronological order, when an inbound video frame associated with this video frame and acquired earlier than this video frame is identified, it indicates that this video frame was acquired after the inbound video frame. This video frame can be an associated video frame (i.e., a video frame acquired during the period from when the moving object enters the target area to when the moving object leaves the target area), or this video frame can be an outbound video frame.

[0076] To distinguish whether a video frame is a related video frame or a departing video frame, the electronic device needs to perform image recognition on the video frame and check whether the image recognition result meets the departing condition. If the image recognition result meets the departing condition, the video frame is determined to be a departing video frame. If the image recognition result does not meet the departing condition, the video frame is determined to be a related video frame.

[0077] Step 1023: If not, perform image recognition on the video frame. If the image recognition result is found to meet the preset entry conditions, then determine that the video frame is the entry video frame.

[0078] In this application embodiment, if an inbound video frame associated with this video frame and acquired earlier than this video frame has not yet been identified, it indicates that the current frame may be an image acquired by the area camera after receiving the acquisition signal and before the moving object enters the target area (such as the background image of the target area) or an inbound video frame that meets the entry conditions.

[0079] To distinguish between these two types of images, before identifying an inbound video frame associated with this video frame and acquired earlier than it, the electronic device can perform image recognition on the current video frame. If the image recognition result meets preset inbound conditions, the current video frame is determined to be the inbound video frame. If the image recognition result does not meet the preset inbound conditions, the current video frame is not processed, and the device directly proceeds to the recognition process of the next video frame.

[0080] Furthermore, in practical applications, as a moving object enters the target area, the proportion of the video frame occupied by the moving object gradually increases until it fills the entire frame or the frame contains the entire moving object. As the moving object leaves the target area, the proportion of the video frame occupied by the moving object gradually decreases. In other words, during the process of the moving object entering the target area, the foreground in the video frame gradually increases, and the background gradually decreases. Conversely, during the process of the moving object leaving the target area, the foreground in the video frame gradually decreases, and the background gradually increases. Therefore, it is evident that the video frame and the background are related. Consequently, this application adds the target background frame determination process described below to more accurately identify the entering and leaving video frames.

[0081] In implementation, steps A1 to A2 need to be executed before step 1021.

[0082] Step A1: The electronic device obtains the motion region in the current video frame; the motion region is the area in the current video frame that has changed compared to the previous video frame.

[0083] In implementation, electronic devices can utilize existing motion region detection algorithms to obtain the motion regions in the current video frame.

[0084] For example, electronic devices can use frame difference methods to detect motion regions in a video frame. Specifically, the electronic device can calculate the frame difference between the current video frame and the previous video frame to find the regions in the current video frame that have changed compared to the previous video frame, and use these as motion regions. Of course, electronic devices can also use background update methods, the ViBe algorithm (a foreground algorithm based on background update), Gaussian mixture modeling algorithms, or deep learning algorithms to obtain the motion regions in the current video frame. This is only an example to illustrate the algorithms for obtaining motion regions, and no specific limitations are imposed on them.

[0085] Step A2: When the proportion of the moving region in this video frame is greater than or equal to a first threshold, if a target background frame for the target region has been recorded, calculate the similarity between the video frame and the target background frame. If the similarity is lower than a preset first similarity threshold, detect whether there is an inbound video frame associated with the video frame and whose acquisition time is earlier than the video frame in at least one video frame. If the target background frame has not been recorded, detect whether there is an inbound video frame associated with the video frame and whose acquisition time is earlier than the video frame in at least one video frame.

[0086] Step A2 will be explained in detail below through steps A21 to A23.

[0087] Step A21: If the proportion of the motion region in this video frame is greater than or equal to the first threshold, the electronic device can detect whether a target background frame targeting the background of the target region is recorded.

[0088] Two situations may be encountered in the embodiments of this application.

[0089] The first scenario is that when the area camera receives the acquisition signal and begins acquisition, the moving object has not yet appeared in the target area. In this case, the area camera acquires an image of the background of the target area. After the area camera has been acquiring background images for a period of time, the moving object enters the target area. In this situation, the electronic device will use the acquired background image of the target area as the target background frame, thus satisfying the condition described above: "a target background frame is recorded."

[0090] The second scenario is that the moving object has already entered the target area when the area camera receives the acquisition signal and begins acquisition. In this case, the electronic device will not record a background frame of the target.

[0091] Step A22: If a background frame for the target area background has been recorded, calculate the similarity between the video frame and the target background frame. If the similarity is lower than a preset first similarity threshold, detect whether an inbound video frame associated with the video frame and whose acquisition time is earlier than the video frame has been identified.

[0092] It should be noted that if a background frame has already been recorded, it indicates the first scenario described above. To enable the electronic device to accurately identify whether a moving object meets the entry or exit conditions, the device calculates the similarity between the target background frame and the current video frame. If the calculated similarity is lower than a first similarity threshold, it indicates that the current video frame differs from the target background frame, which likely means that a moving object has entered the captured image. This means that subsequent identification of the entry, associated, and exit video frames can proceed. Therefore, step 1021 is executed only when the similarity between the target background frame and the current video frame is determined to be lower than the preset first similarity threshold.

[0093] Step A23: If the target background frame is not recorded, check whether an incoming video frame associated with the video frame and whose acquisition time is earlier than the video frame has been identified.

[0094] It should be noted that if no target background frame is recorded, it indicates the second situation mentioned above. In this case, the electronic device can directly execute steps 1021 to 1023.

[0095] In addition, it is necessary to introduce the preset entry conditions and preset exit conditions.

[0096] 1. Preset entry conditions

[0097] The preset entry conditions include: a first condition; or, a first condition and a second condition.

[0098] When the aforementioned target background frame is recorded, the preset conditions may include a first condition and a second condition. When the aforementioned target background frame is not recorded, the preset conditions may include the first condition.

[0099] 1.1) First condition

[0100] In short, the first condition needs to meet two aspects: first, the moving object appears at the entrance of the target area; second, the moving object is moving towards the exit of the target area.

[0101] To achieve these two aspects, in one optional implementation, the first condition can be: the overlapping area between the moving object region and the target region in the video frame is higher than a first region threshold, and the moving object region contains the entry mark region of the target region, and the position of the moving object in the video frame is in a first preset motion direction, wherein the first preset motion direction is formed by the moving object positions identified in the first N frames of the video frame.

[0102] In this context, the first condition, "the overlapping area between the moving object region and the target region in the video frame is higher than a first region threshold, and the moving object region contains the entry marker region of the target region," represents the first aspect mentioned above. The second condition, "the position of the moving object in the video frame is in a first preset motion direction, where the first preset motion direction is formed by the moving object positions identified in the previous N frames of the video frame," represents the second aspect.

[0103] The aforementioned entrance sign area refers to the area where the entrance reference point of the preset target area is located. For example, ... Figure 3 As shown in Figure (a), Figure 3 In Figure (a), the solid rectangle represents the target area, and the right line of the solid rectangle is the entrance marker.

[0104] The threshold for the first region mentioned above can be 1 / 4 of the target region or other values. It can be limited according to the actual situation, but no specific limit is set here.

[0105] The aforementioned first preset motion direction can be the direction formed by the positions of moving objects identified in the previous N frames of the video frame. For example, such as... Figure 3 As shown in Figure b, assuming Figure 3 In (b), the solid-line rectangle represents the target area, the right line of the solid-line rectangle is the entrance marker, and the left line of the solid-line rectangle is the exit marker.

[0106] Assuming the position of the moving object identified in this video frame is as shown by point D, and the positions of the moving objects identified in the first three video frames are as shown by points A, B, and C, points A, B, and C form the motion direction from the entrance to the exit, which is the first preset motion direction.

[0107] For example, such as Figure 3 As shown in Figure (a), Figure 3 In Figure (a), the solid-lined rectangle represents the target area, the right line of the solid-lined rectangle is the entrance marker, and the left line of the solid-lined rectangle is the exit marker. Figure 3 In Figure (a), the dashed rectangle represents the moving object region. When the moving object region (i.e., the dashed rectangle) identified in the video frame covers the right edge of the solid rectangle, and the overlap between the dashed and solid rectangles is greater than 1 / 4 of the area of ​​the solid rectangle, and the direction of motion formed by the positions of the moving objects identified in the previous N frames is from right to left, and the position of the moving object identified in this video frame also moves in this direction from right to left, then the identification result of this video frame meets the entry condition.

[0108] 1.2) Second condition

[0109] The second condition is: the similarity between the video frame and the recorded background frame targeting the target area is greater than or equal to the second similarity threshold.

[0110] The second similarity threshold is less than the first similarity threshold. For example, the second similarity threshold can be 0.8 times the first similarity threshold. This is just an example of the second similarity threshold and is not specifically limited to it.

[0111] It should be noted that the second condition is set to ensure that the video frame and the background frame have a certain degree of difference but also a high degree of similarity. Combined with the first condition, this can effectively characterize the initial features of a moving object entering the target area and effectively detect and identify the event of a moving object entering the target area.

[0112] 2) Preset exit conditions

[0113] The preset exit conditions include: the third condition; or, the second and third conditions.

[0114] When the aforementioned target background frame is recorded, the preset conditions may include the second and third conditions. When the aforementioned target background frame is not recorded, the preset conditions may include the third condition.

[0115] The second condition is as described above and will not be repeated here.

[0116] In simple terms, the third condition needs to satisfy two aspects: first, the moving object appears at the exit of the target area; second, the moving object moves towards the exit of the target area.

[0117] To achieve these two aspects, in one optional implementation, the third condition can be: the overlapping area between the moving object region and the target region in the video frame is lower than the second region threshold and the moving object region contains the exit mark region of the target region, and the position of the moving object in the video frame is in a second preset motion direction, which is formed by the moving object position identified in the first N frames of the video frame.

[0118] In the third condition, "the overlap between the moving object region and the target region in the video frame is less than a first region threshold and the moving object region contains the exit marker region of the target region" represents the first aspect mentioned above. The third condition, "the position of the moving object in the video frame is in a first preset motion direction, which is formed by the moving object positions identified in the previous N frames of the video frame," represents the second aspect.

[0119] The aforementioned exit marking area refers to the area where the pre-defined exit reference point for the target area is located. For example, such as... Figure 4 As shown in Figure (a), Figure 4 In Figure (a), the solid-line rectangle represents the target area, and the left line of the solid-line rectangle is the exit marker.

[0120] The threshold for the second region can be 1 / 4 of the target region or other values, which can be limited according to the actual situation. No specific limitation is made here. The threshold for the second region can be the same as or different from the threshold for the first region; no specific limitation is made here.

[0121] The aforementioned second preset motion direction can be the direction formed by the positions of moving objects identified in the previous N frames of the video frame. For example, such as... Figure 4 As shown in Figure (b), assuming Figure 4 In (b), the solid-line rectangle represents the target area, the right line of the solid-line rectangle is the entrance marker, and the left line of the solid-line rectangle is the exit marker.

[0122] Assuming the position of the moving object identified in this video frame is as shown by point H, and the positions of the moving objects identified in the first three video frames are as shown by points E, F, and G, points E, F, and G form the motion direction from the entrance to the exit, which is the second preset motion direction.

[0123] For example, such as Figure 4 As shown in Figure (a), Figure 4 In Figure (a), the solid-line rectangle represents the target area, the right line of the solid-line rectangle is the entrance marker, and the left line of the solid-line rectangle is the exit marker. Figure 4 In Figure (a), the dashed rectangle represents the moving object region. When the moving object region (i.e., the dashed rectangle) identified in the current video frame covers the left edge of the solid rectangle, and the overlap between the dashed and solid rectangles is less than 1 / 4 of the area of ​​the solid rectangle, and the movement direction formed by the positions of the moving objects identified in the previous N frames is from right to left, and the position of the moving object identified in this video frame is also along this right-to-left movement direction, then the identification result of this video frame meets the exit condition.

[0124] Since the above text mentions background frames, and the background for each vehicle may be different, the issue of updating the target background frame arises.

[0125] The following section discusses the updating of the target background frame and the background frame set.

[0126] In implementation, when the electronic device detects that the proportion of the moving region in the current video frame is less than a first threshold, it performs a target background frame update operation.

[0127] Since the background frame is characterized by minimal changes compared to the previous frame, when the proportion of the detected moving area in this video frame is less than the first threshold, it indicates that this video frame is qualified to be a background frame. Therefore, operations such as setting and updating the target background frame can be performed.

[0128] Specifically, the electronic device can detect whether a background frame is currently being recorded.

[0129] 1) If no target background frame is currently recorded, then the video frame is determined as the target background frame and added to the background frame set.

[0130] 2) If a target background frame has been recorded, the recorded target background frame and the background frame set to which the target background frame belongs are updated based on the similarity between the video frame and at least one background frame in the set of recorded background frames.

[0131] In implementation, the electronic device determines the similarity between the video frame and background frames in the recorded background frame set. For example, the electronic device can determine the similarity between the video frame and each background frame in the background frame set. Of course, the electronic device can also determine the similarity between the video frame and multiple background frames in the background frame set; no specific limitation is made here.

[0132] In one alternative implementation, when the determined maximum similarity is greater than a third similarity threshold, the electronic device updates the currently recorded target background frame to the current video frame. The third similarity threshold can be the same as the first similarity threshold mentioned above.

[0133] In another optional implementation, although the maximum similarity between this video frame and multiple background frames in the background frame set is greater than the third similarity threshold, this video frame may be a scene captured when a moving object enters the target area but has not yet met the entry conditions. Strictly speaking, this type of scene belongs to pseudo background and is not a true target background frame with only background. Therefore, in the preferred implementation, if this video frame is such a pseudo background frame, the target background frame update operation is not performed. The target background frame update operation is performed only when it is not such a pseudo background frame.

[0134] In implementation, if the determined maximum similarity is greater than the third similarity threshold, and the recognition result of the video frame does not meet the target conditions, then the target background frame will be updated. If the recognition result of the video frame meets the target conditions, then the target background frame will not be updated.

[0135] The target condition is as follows: the overlap between the moving object region and the target region in the video frame is less than a first region threshold, and the moving object region contains the entry marker region of the target region.

[0136] The threshold for the first region is 1 / 4 of the target region. This is just an example of the threshold for the first region and is not specifically limited to it.

[0137] Furthermore, in this embodiment of the application, when the similarity between the current video frame and N target background frames in the background frame set is less than the fourth similarity threshold, it indicates that the current video frame is significantly different from the previous target background frames. Therefore, the electronic device can maintain the recorded target background frames unchanged.

[0138] Among them, the fourth similarity threshold is less than the third similarity threshold, and the fourth similarity threshold can be 0.5 times the third similarity threshold.

[0139] Since updating the target background frame involves the background frame set, it also involves the issue of updating the background frame set.

[0140] In this embodiment of the application, the electronic device can update the background frame set based on the similarity between the current video frame and the background frames in the background frame set calculated above.

[0141] In implementation, if the calculated maximum similarity is greater than the third similarity threshold, it indicates that the current video frame is very similar to the background frame used to calculate the maximum similarity, and the current video frame is newer than the background frame used to calculate the maximum similarity. In this case, the electronic device will update the background frame used to calculate the maximum similarity to the current video frame.

[0142] For example, suppose the background frame set includes: background frame 1, background frame 2, and background frame 3. Suppose the similarity between this video frame and background frame 1 is 65%, the similarity between this video frame and background frame 2 is 70%, and the similarity between this video frame and background frame 3 is 90%. Suppose the third similarity threshold is 85%. Since this video frame has the highest similarity to background frame 3, and this highest similarity is greater than the third similarity threshold (i.e., 85%), the electronic device will replace background frame 3 in the background frame set with this video frame.

[0143] Among them, background frame 3 is referred to as "the target background frame used to calculate the maximum similarity".

[0144] Furthermore, if the similarity between this video frame and N background frames in the background frame set is less than the fourth similarity threshold, it indicates that this video frame is not very similar to the background frames in the background frame set, so the video frame should be added to the background frame set.

[0145] In addition, to prevent the background frame set from taking up too much storage space, electronic devices will also delete some background frames that are far removed from the current time.

[0146] In implementation, before updating the background frame set, the electronic device also needs to score the video frame. When the background frame deletion condition is met, the M background frames with the lowest scores in the background frame set are deleted. The M background frames with the lowest scores represent target background frames that are relatively far from the current time, i.e., older background frames.

[0147] The following steps, B1 to B2, provide a detailed explanation of "determining the score of the video frame".

[0148] Step B1: The electronic device determines the target value associated with the video frame.

[0149] In implementation, the electronic device can check whether the recognition result obtained by image recognition of the video frame meets the preset target value determination conditions.

[0150] If the recognition result meets a preset target value determination condition, the electronic device determines the target value associated with the video frame based on the target value indicated by the preset target value determination condition;

[0151] If the recognition result satisfies two or more preset target value determination conditions, the electronic device determines the target value associated with the video frame based on the target value indicated by the preset target value determination condition with the highest priority.

[0152] The preset target value determination conditions include at least: a first preset target value determination condition, a second preset target value determination condition, and a third preset target value determination condition.

[0153] 1) Conditions for determining the first preset target value

[0154] The first preset target value determination condition is used to indicate that the target value is n1 when the proportion of the motion region in the video frame is less than or equal to the second threshold.

[0155] The motion region is the area in the video frame that has changed compared to the previous video frame. The second threshold is less than the first threshold; for example, if the first threshold is 10%, the second threshold could be 8%.

[0156] 2) Conditions for determining the second preset target value

[0157] The second preset target value determination condition is used to indicate a target value of n2 when the overlapping area between the moving object region and the target region in the video frame is less than the first region threshold and the moving object region contains the entry mark region of the target region.

[0158] Assuming the first region threshold is 1 / 4 of the target region, the second preset target value determination condition indicates that when a moving object enters the target region but has not yet entered 1 / 4 of the target region (or in other words, the moving object enters the target region but has not yet met the entry condition), the target value is indicated as n2.

[0159] 3) Conditions for determining the third preset target value

[0160] The third preset target value determination condition is used to indicate that the target value is n3 when the recognition result of the video frame meets the exit condition.

[0161] It should be noted that the priority of the first preset target value determination condition is lower than the priority of the second preset target value determination condition and the third preset target value determination condition.

[0162] Furthermore, it should be noted that since the background frame set needs to maintain the background frame closest to the current time, and the background frame usually does not contain the background image of the target area of ​​the moving object (e.g., the image captured between the time the previous moving object leaves the target area and the time the next moving object enters the target area), this means that the closer the moving object is to the exit of the target area, the more likely a background frame will be captured. Conversely, the entry of the moving object into the target area indicates that a background frame will not be captured until some time has passed after the moving object leaves. Based on the above reasons, the magnitude relationship of the four target values ​​can be set as: n2 ≤ n1 ≤ n3.

[0163] Step B2: The electronic device determines the video frame score of the video frame based on the score of the previous target background frame added to the background frame set and the determined target value.

[0164] In one alternative implementation, after determining the target value associated with the current video frame, the electronic device can add the score of the previous target background frame added to the background frame set to the target value associated with the current video frame to obtain the score of the current video frame.

[0165] In this embodiment of the application, when a background frame deletion condition is detected, the electronic device deletes the M target background frames with the lowest scores from the background frame set; M is an integer greater than 0.

[0166] The conditions for deleting a background frame can be that the total number of background frames in the background frame set is greater than a preset threshold, the background frame has not been updated for a long time, the update time of the background frame has arrived, or a new background frame has been added.

[0167] Step 103: The electronic device can stitch together the incoming video frame, the associated video frame, and the outgoing video frame.

[0168] The following section will provide a detailed introduction to step 103 from two aspects: the timing of splicing and the splicing method.

[0169] 1) Timing of splicing

[0170] In one alternative implementation, when the electronic device identifies a related video frame, it can stitch the related video frame with the previously stitched image, which is a stitched image of the incoming video frame and the previously identified related video frame. When the electronic device identifies an outgoing video frame, it can stitch the outgoing video frame with the previously recorded stitched image to obtain a global image of the moving object.

[0171] In simple terms, when the electronic device identifies the first associated video frame, it stitches the first associated video frame with the incoming video frame. When the electronic device identifies the second associated video frame, it stitches the second associated video frame with the stitched image formed by ...

[0172] The advantage of this stitching method is that it stitches the video frames after each associated or outgoing video frame is identified, achieving a real-time stitching effect and quickly generating a global map of the moving object.

[0173] In another alternative implementation, after identifying the exiting video frame, the electronic device stitches together the entering video frame, the associated video frame, and the exiting video frame.

[0174] This is merely an illustrative example of the timing of splicing, and does not specifically limit the timing of splicing.

[0175] 2) Splicing method

[0176] In practice, electronic devices can use existing splicing methods to complete the above splicing.

[0177] For example, an electronic device can identify a first moving object region from a first image to be stitched together. The electronic device can identify a second moving object region from a second image to be stitched together.

[0178] Then, the electronic device can preprocess the first moving object region and the second moving object region to obtain a more complete moving object region. For example, the electronic device can perform noise reduction processing on the first moving object region and the second moving object region respectively.

[0179] Then, the electronic device can use the difference between the first moving object region and the second moving object region as a mask, and use the mask and the speed of the moving object to determine the length of the overlapping region of the first moving object region and the second moving object region.

[0180] Finally, electronic devices can combine the length of the overlapping areas and use a fade-in / fade-out blending method to splice them together.

[0181] For example, when an electronic device uses the first splicing method described above, the splicing formula is as follows:

[0182]

[0183] in, Determined by the length of the overlapping region;

[0184] This represents the stitched image obtained from the previous stitching process;

[0185] Indicates the region of the moving object indicated by the current video frame;

[0186] This represents the stitched image obtained from this stitching process.

[0187] As described above, this application uses image recognition technology to identify, from at least one video frame of the target area, the video frame entering the target area, the associated video frame and the video frame exiting the target area, thereby identifying multiple video frames to be stitched together from the entry of the moving object into the target area to the exit of the target area.

[0188] As can be seen from the above description, on the one hand, compared with acquiring the video frames to be stitched through sensors, the method of this application does not require the configuration of additional inlet and outlet sensors, so the method of this application can greatly reduce the hardware deployment cost;

[0189] On the other hand, since this application does not use video frames based on the time between entry and exit as the video frames to be stitched in the traditional way, but directly identifies the entry video frames based on image recognition, the entry video frames identified by this application can more accurately reflect the moving object entering the target area, the associated video frames can more accurately reflect the moving object passing through the target area, and the exit video frames can more accurately reflect the moving object leaving the target area. Therefore, the global map stitched from these three types of video frames can more accurately and realistically reflect the overall situation of the moving object.

[0190] The above splicing method will be explained in detail below using a vehicle as an example of the moving object.

[0191] In this scenario, a front-facing camera is positioned outside the target area, a body-mounted camera is positioned inside the target area, and a rear-facing camera is also deployed within the target area. The front-facing camera captures images of the front of the vehicle, the rear-facing camera captures images of the rear of the vehicle, and the body-mounted camera captures images of the body of the vehicle.

[0192] When the front-facing camera detects the front of the vehicle, it sends a capture signal to the body-mounted camera, which then captures an image of the target area, forming at least one video frame.

[0193] See Figure 5 , Figure 5 This is a flowchart illustrating another stitching method in an exemplary embodiment of this application. The method can be applied to a vehicle body camera and may include the steps shown below.

[0194] Step 501: Obtain a video frame to be identified from at least one video frame;

[0195] Step 502: Identify the motion region of the video frame.

[0196] For specific identification methods, please refer to step A1 above, which will not be repeated here.

[0197] Step 503: Detect whether the proportion of the moving region in the video frame is greater than or equal to the first threshold;

[0198] If not, proceed to step 504, and then return to step 501 after completing step 504.

[0199] If so, proceed to step 505.

[0200] Step 504: Perform update processing for the target background frame and the set of background frames containing the target background frame;

[0201] For an introduction to background frame update processing, please refer to the background frame update introduction above, which will not be repeated here.

[0202] Step 505: Detect whether a target background frame is recorded;

[0203] If so, proceed to step 506;

[0204] If not, proceed to step 508.

[0205] Step 506: Calculate the similarity between the video frame and the target background frame;

[0206] Step 507: Detect whether the calculated similarity is greater than the first similarity threshold;

[0207] If so, return to step 501;

[0208] If not, proceed to step 508.

[0209] Step 508: Detect whether an incoming video frame associated with this video frame and whose acquisition time is earlier than this video frame has been identified;

[0210] If so, proceed to step 511;

[0211] If not, proceed to step 509.

[0212] Step 509: Check whether the video frame meets the entry conditions;

[0213] If so, proceed to step 510;

[0214] If not, return to step 501.

[0215] Step 510: Determine that the video frame is an inbound video frame;

[0216] Step 511: Does the image recognition result of this video frame meet the preset exit conditions?

[0217] If so, proceed to step 513;

[0218] If not, proceed to step 512.

[0219] Step 512: Determine that the video frame is a related video frame, and stitch together the stitched image obtained from the previous stitching of the video frame or the incoming video frame;

[0220] Step 513: Determine that the video frame is the exit video frame, and stitch the exit video frame with the stitched image obtained in the previous stitching to obtain the global image of the vehicle body.

[0221] Furthermore, the business server can determine the vehicle type based on the front image captured by the front camera, the overall vehicle image stitched together from the body cameras, and the rear image captured by the rear camera, and then perform subsequent business processing based on the vehicle type. For example, it can determine the payment amount based on the vehicle type. This will not be elaborated further here.

[0222] See Figure 6 , Figure 6 This is a hardware structure diagram of an electronic device illustrated in an exemplary embodiment of this application.

[0223] The electronic device includes a communication interface 601, a processor 602, a machine-readable storage medium 603, and a bus 604; wherein the communication interface 601, the processor 602, and the machine-readable storage medium 603 communicate with each other via the bus 604. The processor 602 can execute the image stitching method described above by reading and executing machine-executable instructions corresponding to the image stitching control logic in the machine-readable storage medium 603.

[0224] The machine-readable storage medium 603 mentioned herein can be any electronic, magnetic, optical, or other physical storage device that can contain or store information, such as executable instructions, data, etc. For example, the machine-readable storage medium can be volatile memory, non-volatile memory, or similar storage media. Specifically, the machine-readable storage medium 603 can be RAM (Random Access Memory), flash memory, a storage drive (such as a hard disk drive), a solid-state drive, any type of storage disk (such as an optical disc, DVD, etc.), or similar storage media, or combinations thereof.

[0225] See Figure 7 , Figure 7This is a block diagram of an image stitching device according to an exemplary embodiment of this application. The device can be applied to an electronic device and may include the following units:

[0226] Acquisition unit 701 is used to acquire at least one video frame captured for the target area;

[0227] The identification unit 702 is configured to identify an entering video frame, an associated video frame, and an exit video frame from the at least one video frame; wherein the entering video frame, the associated video frame, and the exit video frame contain the same moving object; the entering video frame is an image captured when the moving object enters the target area, the associated video frame is an image captured after the moving object enters the target area and before it exits the target area, and the exit video frame is an image captured when the moving object exits the target area;

[0228] The splicing unit 703 is used to splice the incoming video frame, the associated video frame, and the outgoing video frame.

[0229] Optionally, when identifying an entering video frame, an associated video frame associated with the entering video frame, and an exit video frame from at least one video frame, the identification unit 702 is configured to determine an entering video frame that meets preset entering conditions from at least one video frame; determine an exit video frame that meets preset exit conditions from at least one video frame; and use the video frame between the entering video frame and the exit video frame as an associated video frame.

[0230] Optionally, the identification unit 702, when determining an entering video frame that meets preset entering conditions from at least one video frame; determining an exit video frame that meets preset exit conditions from at least one video frame; and using video frames between the entering and exit video frames as associated video frames, is configured to, for each of the at least one video frames, detect whether an entering video frame associated with the video frame and whose acquisition time is earlier than the video frame has been identified; if so, perform image recognition on the video frame; if the recognition result does not meet the preset exit conditions, determine the video frame as an associated video frame associated with the entering video frame; if the image recognition result of the video frame meets the exit conditions, determine the video frame as an associated exit video frame associated with the entering video frame; if not, perform image recognition on the video frame; if the image recognition result meets the preset entering conditions, determine the video frame as the entering video frame.

[0231] Optionally, the identification unit 702, when detecting whether an inbound video frame associated with the video frame and acquired earlier than the video frame has been identified, is used to obtain a motion region in the video frame; the motion region is the region in the video frame that has changed compared to the previous video frame; if the proportion of the motion region in the video frame is greater than or equal to a first threshold, then if a target background frame for the target region background has been recorded, the similarity between the video frame and the target background frame is calculated; if the similarity is lower than a preset first similarity threshold, it is detected whether an inbound video frame associated with the video frame and acquired earlier than the video frame has been identified; if the target background frame has not been recorded, it is detected whether an inbound video frame associated with the video frame and acquired earlier than the video frame has been identified.

[0232] Optionally, if a target background frame has been recorded, the entry condition includes a first condition and a second condition; the exit condition includes a second condition and a third condition.

[0233] In the absence of a recorded target background frame, the entry condition includes a first condition, and the exit condition includes a third condition.

[0234] The first condition is: the overlapping area between the moving object region and the target region in the video frame is higher than the first region threshold and the moving object region contains the entry mark region of the target region, and the position of the moving object in the video frame is in a first preset motion direction, wherein the first preset motion direction is formed by the moving object position identified in the first N frames of the video frame;

[0235] The second condition is: the similarity between the video frame and the recorded target background frame for the target area background is greater than or equal to the second similarity threshold;

[0236] The third condition is: the overlap between the moving object region and the target region in the video frame is lower than the second region threshold, and the moving object region contains the exit mark region of the target region, and the position of the moving object in the video frame is in the second preset motion direction; the second preset motion direction is formed by the moving object position identified in the first N frames of the video frame.

[0237] Optionally, the device further includes:

[0238] The update unit 704 is used to update the recorded target background frame and the background frame set to which the target background frame is located, based on the similarity between the video frame and at least one background frame in the recorded background frame set when the proportion of the motion region in the video frame is less than a first threshold.

[0239] Optionally, when updating a recorded target background frame based on the similarity between the video frame and at least one background frame in the recorded background frame set, the updating unit 704 is used to determine the similarity between the video frame and at least one background frame in the recorded background frame set; if the determined maximum similarity is greater than a third similarity threshold, the recorded target background frame is updated to the video frame; if the similarity between the video frame and N target background frames in the background frame set is less than a fourth similarity threshold, the recorded target background frame is kept unchanged; wherein, the fourth similarity threshold is less than the third similarity threshold.

[0240] Optionally, when updating the recorded target background frame, the updating unit 704 is used to update the target background frame when the recognition result of the video frame does not meet the target conditions; wherein, the target conditions are: the overlapping area between the moving object region and the target region in the video frame is less than a first region threshold, and the moving object region contains the entry mark region of the target region.

[0241] Optionally, when updating the background frame set containing the target background frame, the updating unit 704 is configured to replace the background frame used to calculate the maximum similarity with the video frame if the determined maximum similarity is greater than the third similarity threshold; and add the video frame to the background frame set if the similarity between the video frame and N target background frames in the background frame set is less than the fourth similarity threshold; wherein N is an integer greater than 1, and the third similarity threshold is greater than the fourth similarity threshold.

[0242] Optionally, before updating the background frame set, the updating unit 704 is further configured to determine the target value associated with the video frame; and determine the video frame score of the video frame based on the score corresponding to the previous background frame added to the background frame set and the determined target value.

[0243] The update unit 704, when updating the background frame set to which the target background frame is located, is used to delete the M background frames with the lowest scores from the background frame set when the background frame deletion condition is met; M is an integer greater than 0.

[0244] Optionally, when determining the target value associated with the video frame, the updating unit 704 checks whether the recognition result obtained by image recognition of the video frame meets a preset target value determination condition; if the recognition result meets one preset target value determination condition, the target value associated with the video frame is determined according to the target value indicated by the preset target value determination condition; if the recognition result meets two or more preset target value determination conditions, the target value associated with the video frame is determined according to the target value indicated by the preset target value determination condition with the highest priority.

[0245] The conditions for determining the preset target value include at least the following:

[0246] The first preset target value determination condition is used to indicate a target value of n1 when the proportion of the motion region in the video frame is less than or equal to the second threshold; the motion region is the region in the video frame that has changed compared to the previous video frame.

[0247] The second preset target value determination condition is used to indicate a target value of n2 when the overlapping area between the moving object region and the target region in the video frame is less than the first region threshold and the moving object region contains the entry mark region of the target region.

[0248] The third preset target value determination condition is used to indicate that the target value is n3 when the recognition result of the video frame meets the exit condition;

[0249] Among them, the priority of the first preset target value determination condition is lower than the priority of the second preset target value determination condition and the third preset target value determination condition;

[0250] The n2≤n1≤n3.

[0251] In addition, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described image stitching method.

[0252] In addition, this application also provides a computer program stored in a computer-readable storage medium, which, when executed by a processor, causes the processor to implement the above-described image stitching method.

[0253] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0254] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0255] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. An image stitching method applied in a high-speed payment or parking lot payment scene, characterized in that, The method comprises: obtaining at least one video frame collected for a target region; the target region comprises an entry sign region and an exit sign region; the entry sign region refers to a region where an entry reference object of the target region is located; the exit sign region refers to a region where an exit reference object of the target region is located; identifying, from the at least one video frame, an entry video frame, an associated video frame associated with the entry video frame, and an exit video frame; wherein the entry video frame, the associated video frame, and the exit video frame have a same moving object; the entry video frame is an image collected when the moving object enters the target region, the associated video frame is an image collected before the moving object exits the target region after entering the target region, and the exit video frame is an image collected when the moving object exits the target region; stitching the entry video frame, the associated video frame, and the exit video frame to generate a global image of the moving object; wherein identifying, from the at least one video frame, the entry video frame, the associated video frame, and the exit video frame comprises: determining, from the at least one video frame, an entry video frame that satisfies a preset entry condition, and determining, from the at least one video frame, an exit video frame that satisfies a preset exit condition; and regarding a video frame between the entry video frame and the exit video frame as the associated video frame; in a case where a target background frame has been recorded, the entry condition comprises a first condition and a second condition, and the exit condition comprises the second condition and a third condition; in a case where the target background frame has not been recorded, the entry condition comprises the first condition, and the exit condition comprises the third condition; the first condition is that an overlap area of a moving object region in a video frame with the target region is higher than a first area threshold, the moving object region contains the entry sign region of the target region, and a position of the moving object in the video frame is in a first preset moving direction formed by positions of the moving object identified from previous N video frames; the second condition is that a similarity between the video frame and a target background frame recorded for a background of the target region is greater than or equal to a second similarity threshold; and the third condition is that the overlap area of the moving object region in the video frame with the target region is lower than a second area threshold, the moving object region contains the exit sign region of the target region, and the position of the moving object in the video frame is in a second preset moving direction formed by the positions of the moving object identified from the previous N video frames; The splicing comprises: in the order of video frame acquisition, for each two adjacent to-be-spliced video frames, performing the following steps: generating a mask according to the difference of the moving object regions in the two adjacent to-be-spliced video frames, determining the length of the overlap region of the moving object regions according to the mask and the speed of the moving object, and calculating a splicing weight based on the length of the overlap region, and splicing the splicing result of the current video frame and the previous video frame based on the splicing weight by using a crossfade fusion manner; the two adjacent to-be-spliced video frames include the current video frame and the corresponding previous video frame.

2. The method of claim 1, wherein, The driving-in video frame and the driving-out video frame are determined from at least one video frame, and the video frames between the driving-in video frame and the driving-out video frame are determined as the associated video frames, comprising: For each video frame in the at least one video frame, detecting whether a driving-in video frame associated with the video frame and collected earlier than the video frame has been identified; If yes, performing image recognition on the video frame, and if it is detected that the recognition result does not meet the preset driving-out condition, determining that the video frame is the associated video frame associated with the driving-in video frame; if the image recognition result of the video frame meets the driving-out condition, determining that the video frame is the driving-out video frame associated with the driving-in video frame; If no, performing image recognition on the video frame, and if it is detected that the image recognition result meets the preset driving-in condition, determining that the video frame is the driving-in video frame.

3. The method of claim 2, wherein, The detection whether a driving-in video frame associated with the video frame and collected earlier than the video frame has been identified comprises: Obtaining a motion region in the video frame; the motion region is a region in the video frame that changes compared with a previous video frame; If the proportion of the motion region in the video frame is greater than or equal to a first threshold value, then: If a target background frame for a target region background has been recorded, calculating the similarity between the video frame and the target background frame, and when the similarity is lower than a preset first similarity threshold value, detecting whether a driving-in video frame associated with the video frame and collected earlier than the video frame has been identified; If the target background frame has not been recorded, detecting whether a driving-in video frame associated with the video frame and collected earlier than the video frame has been identified.

4. The method of claim 3, wherein, When the proportion of the motion region in the video frame is less than the first threshold value, the method further comprises: According to the similarity between the video frame and at least one background frame in the recorded background frame set, updating the recorded target background frame and updating the background frame set in which the target background frame is located.

5. The method of claim 4, wherein, The updating the recorded target background frame according to the similarity between the video frame and at least one background frame in the recorded background frame set comprises: Determining the similarity between the video frame and at least one background frame in the recorded background frame set; If the determined maximum similarity is greater than a third similarity threshold value, updating the recorded target background frame to the video frame; If the similarity between the video frame and each of the N target background frames in the background frame set is less than a fourth similarity threshold, the recorded target background frame is maintained unchanged. The fourth similarity threshold is less than the third similarity threshold.

6. The method of claim 5, wherein, The updating of the recorded target background frame comprises: The target background frame is updated when the identification result of the video frame does not satisfy a target condition. The target condition is that the overlap between the moving object region in the video frame and the target region is less than a first region threshold, and the moving object region contains an entrance mark region of the target region.

7. The method of claim 5, wherein, The updating of the background frame set in which the target background frame is located comprises: If the determined maximum similarity is greater than a third similarity threshold, the background frame used to calculate the maximum similarity is replaced by the video frame. If the similarity between the video frame and each of the N target background frames in the background frame set is less than a fourth similarity threshold, the video frame is added to the background frame set; N is an integer greater than 1, and the third similarity threshold is greater than the fourth similarity threshold.

8. The method of claim 4, wherein, Before the updating of the background frame set, the method further comprises: Determining a target value associated with the video frame; Determining a video frame score of the video frame according to the score corresponding to the background frame added to the background frame set last time and the determined target value. The updating of the background frame set in which the target background frame is located further comprises: When a background frame deletion condition is satisfied, M background frames with the lowest scores are deleted from the background frame set; M is an integer greater than 0.

9. The method of claim 8, wherein, The determination of the target value associated with the video frame comprises: Checking a preset target value determination condition satisfied by an identification result obtained by performing image recognition on the video frame; If the identification result satisfies one preset target value determination condition, determining the target value associated with the video frame according to the target value indicated by the preset target value determination condition; If the identification result satisfies two or more preset target value determination conditions, determining the target value associated with the video frame according to the target value indicated by the preset target value determination condition with the highest priority; The preset target value determination conditions comprise at least: A first preset target value determination condition for indicating a target value n1 when the proportion of a moving region in the video frame is less than or equal to a second threshold; the moving region is a region in the video frame that has changed compared with a previous video frame; A second preset target value determination condition for indicating a target value n2 when the overlap between a moving object region in the video frame and the target region is less than a first region threshold, and the moving object region contains an entrance mark region of the target region; A third preset target value determination condition for indicating the target value n3 when the identification result of the video frame satisfies the driving-out condition. The priority of the first preset target value determination condition is lower than that of the second preset target value determination condition and the third preset target value determination condition. The n2≤n1≤n3.

10. An image stitching apparatus characterized by comprising: In the application to the high-speed payment or parking lot payment scenarios, the device comprises: The acquisition unit is configured to obtain at least one video frame collected for a target region; the target region includes an entrance sign region and an exit sign region; the entrance sign region refers to a region where an entrance reference object of the target region is located; the exit sign region refers to a region where an exit reference object of the target region is located; The identification unit is configured to identify, from the at least one video frame, an entering video frame, an associated video frame associated with the entering video frame, and an exiting video frame; the entering video frame and the associated video frame and the exiting video frame have a same moving object; the entering video frame is an image collected when the moving object enters the target region, the associated video frame is an image collected before the moving object exits the target region after entering the target region, and the exiting video frame is an image collected when the moving object exits the target region; The splicing unit is configured to splice the entering video frame, the associated video frame, and the exiting video frame to generate a global image of the moving object. The identification unit is specifically configured to determine, from the at least one video frame, an entering video frame that satisfies a preset entering condition, determine, from the at least one video frame, an exiting video frame that satisfies a preset exiting condition, and take video frames between the entering video frame and the exiting video frame as the associated video frame. In a case where a target background frame has been recorded, the entering condition includes a first condition and a second condition, and the exiting condition includes the second condition and a third condition; in a case where the target background frame has not been recorded, the entering condition includes the first condition, and the exiting condition includes the third condition. The first condition is that an overlap region of a moving object region in a video frame and the target region is higher than a first region threshold, the moving object region includes the entrance sign region of the target region, and a position of the moving object in the video frame is in a first preset moving direction formed by positions of the moving object identified from the first N frames of the video frame; the second condition is that a similarity between the video frame and a target background frame recorded for a background of the target region is greater than or equal to a second similarity threshold. The third condition is that an overlap region of a moving object region in a video frame and the target region is lower than a second region threshold, the moving object region includes the exit sign region of the target region, and a position of the moving object in the video frame is in a second preset moving direction formed by positions of the moving object identified from the first N frames of the video frame. The splicing unit is specifically configured to: in accordance with a video frame acquisition sequence, for each two adjacent to-be-spliced video frames, performing the following steps: generating a mask according to a difference of a motion object region in the two adjacent to-be-spliced video frames, determining a length of an overlap region of the motion object region according to the mask and a speed of the motion object, and calculating a splicing weight based on the length of the overlap region, and splicing a splicing result of a current video frame and a previous video frame based on the splicing weight by using a cross-fade fusion manner; the two adjacent to-be-spliced video frames include the current video frame and the corresponding previous video frame.

11. An electronic device, comprising: The electronic device includes a readable storage medium and a processor; The readable storage medium is configured to store machine executable instructions. The processor is configured to read the machine executable instructions on the readable storage medium and execute the instructions to implement the steps of the method in any one of claims 1-9.

Images