Video generation method and apparatus, and detection system and video playback device

Abstract

This application provides a video generation method and apparatus, as well as a detection system and video playback device. Based on this application, a visible light image that fully presents a continuous transmission path through a transmission scanning channel can be obtained. Furthermore, the structural features of the target object can be fused into the image sequence of the visible light image. This allows the inspection video generated based on the image sequence of the visible light image to display the continuous, uninterrupted movement of the target object along the continuous transmission path in a way that correlates the structural and shape features of the target object. Thus, the correlated presentation of the structural and shape features of the target object in the inspection video helps inspection personnel improve the efficiency of object matching in identifying target objects among physical objects. In addition, this application can also automatically identify abnormal target objects through intelligent feature recognition and dynamically display visual abnormality alarm tags in the inspection video.

Description

Technical Field

[0001] This application relates to object detection technology, and in particular to a video generation method, a video generation device, a detection system, and a video playback device that can realize a hybrid presentation of transmission imaging and visible light imaging. Background Technology

[0002] In detection systems based on transmitted rays (e.g., X-rays), the transmission scanning component that generates the transmitted rays is typically deployed within a transmission scanning channel with a closed, shielded enclosure to prevent radiation contamination of the scene where the detection system is located. The detection system also includes a transport mechanism for conveying the target object, allowing it to pass through the transmission scanning channel and be scanned by the transmitted rays generated by the transmission scanning component within the channel, thereby acquiring a transmitted ray image (e.g., an X-ray image) of the target object obtained by the transmission scanning component.

[0003] Transmitted X-ray images contain image content characterizing the structural features of a target object. By displaying these images on a display device, inspectors can determine whether the target object contains any abnormalities. For example, in security inspection applications such as pedestrian traffic or logistics sorting, the target object could be a portable bag carried by a pedestrian or a package awaiting sorting. By observing the transmitted X-ray image displayed on the display device, inspectors can determine whether the portable bag or package contains dangerous or harmful items that endanger personal safety, based on the structural features shown in the transmitted X-ray image. As another example, in quality inspection applications such as parts manufacturing or food processing, the target object could be industrial parts or food. By observing the transmitted X-ray image displayed on the display device, inspectors can determine whether the industrial parts have structural defects or whether the food contains inedible foreign objects, based on the structural features shown in the transmitted X-ray image.

[0004] Furthermore, since the transmitted X-ray image only includes the structural features of the target object, when any target object is identified as abnormal by observing the transmitted X-ray image, the inspector needs to further observe the shape features of the entity object that has been transmitted out of the transmission scanning channel. Moreover, based on the observation and comparison of the structural features of the target object and the shape features of the entity object, object matching is achieved between the target object presented in the transmitted X-ray image and the entity object existing in the scene.

[0005] However, object matching based on human visual observation is inefficient and not conducive to searching and locating abnormal target objects among numerous entities, thus delaying subsequent processing of abnormal target objects. If target objects such as portable bags or packages are taken away from the conveyor by their owners, or if target objects such as industrial parts or food are conveyed by the conveyor to downstream processes such as product packaging, the optimal time for processing abnormal objects may be missed.

[0006] Therefore, how to improve the object matching efficiency in the detection process based on transmitted rays has become a technical problem that needs to be solved in the existing technology. Summary of the Invention

[0007] In view of this, the embodiments of this application provide a video generation method, a video generation device, a detection system, and a video playback device that can realize the mixed presentation of transmission imaging and visible light imaging, which helps to improve the object matching efficiency in the detection process based on transmission rays.

[0008] In one embodiment of this application, a video generation method includes:

[0009] Visible light images are generated using image acquisition data produced by a visible light imaging component. The visible light imaging component is configured to image the channel segments inside the transmission scanning channel and the channel segments outside the entrance and exit of the transmission scanning channel. The visible light images include continuous transmission segments where the channel segments inside and the channel segments outside are connected to each other. Furthermore, an over-detection video generated based on the image sequence of the visible light images is used to present the movement process of the target object along the continuous transmission segments.

[0010] Using transmission scanning components arranged inside the transmission scanning channel to generate transmission scanning data of the target object passing through the segment within the channel, the structural features of the target object are fused into the image sequence of the visible light image, wherein the structural features are matched with the position of the shape features of the target object included in the visible light image, so that the continuous movement process presented in the over-detection video includes: a correlated presentation process in which the structural features and the shape features gradually change synchronously in response to the continuous movement of the target object.

[0011] In some examples, optionally, the visible light imaging component includes a first camera component, a second camera component, and a third camera component, wherein the imaging field of view of the first camera component covers the channel segment, and the imaging fields of view of the second and third camera components respectively cover the channel segment outside the entrance and exit of the transmission scanning channel; the image acquisition data generated by the visible light imaging component includes the channel image sequence generated by the first camera component and the channel image sequence generated by the second and third camera components; generating a visible light image using the image acquisition data generated by the visible light imaging component includes: generating the visible light image based on the stitching of synchronous frames in the channel image sequence and the channel image sequence.

[0012] In some examples, optionally, fusing the structural features of the target object in the image sequence of the visible light image includes: fusing frame-by-frame the structural features of the target object that have been transmissively scanned by the transmission scanning component and are still within the channel segment in the visible light image at the imaging time within the fusion time window, wherein the fusion time window is used to characterize the time range from the target object arriving at the scanning position of the transmission scanning component to the target object moving out of the channel segment, and the associated presentation process of the structural features and the shape features is configured such that: during the period when any part of the target object becomes the scanned part of interest, the presentation state of the part in the over-inspection video switches from being highlighted by the corresponding part shape features to being highlighted by the part structural features, and the presentation state of the part switches back to being highlighted by the corresponding part shape features in response to a change in position after moving out of the channel segment.

[0013] In some examples, optionally, the method further includes: when the target object is detected on the outer segment of the channel outside the entrance of the transmission scanning channel, determining the fusion time window based on the transmission speed of the transmission mechanism and the distance of the target object's current position relative to the scanning position.

[0014] Optionally, in some examples, the method further includes: when it is detected that the data content contained in the transmission scan data changes from the full-load background correction data of the road segment within the channel to the feature content of the structural feature, the fusion of the structural feature of the location is initiated.

[0015] Optionally, in some examples, the method further includes stopping the fusion of the structural features of the location when it is detected that the road segment within the channel becomes empty due to the removal of the target object.

[0016] In some examples, optionally, the part construction features are dynamically refreshed in response to continuous movement of the target object; in the visible light image where the imaging time is within the fusion time window, the part construction features of the target object that have been transmissively scanned by the transmission scanning component and are still within the channel segment are fused frame by frame, including: based on the time matching between the refresh time of the dynamic refresh and the imaging time of the visible light image, the dynamically refreshed part construction features are fused into the visible light image, so that the fused part construction features in the visible light image are refreshed at a refresh time that matches the imaging time of the visible light image.

[0017] In some examples, optionally, the over-detected video is generated by encoding the image sequence of the visible light image frame by frame; based on the time matching between the refresh time of the dynamic refresh and the imaging time of the visible light image, the dynamically refreshed particulate structural features are fused into the visible light image, including: in response to the arrival of the encoding time of each frame of the visible light image, determining a matching refresh time that matches the imaging time of the visible light image of the current frame to be encoded; and fusing the particulate structural features that have been refreshed at the matching refresh time into the visible light image of the current frame to be encoded.

[0018] In some examples, optionally, the site construction features are dynamically refreshed at the scanning frequency of the transmission scanning component, and the frame rate of the visible light image sequence is different from the scanning frequency; determining a matching refresh time that matches the imaging time of the visible light image of the current frame to be encoded includes: in the refresh time sequence that matches the scanning frequency, determining the refresh time that is closest to the imaging time of the visible light image of the current frame to be encoded as the matching refresh time, wherein if the matching refresh time is later than the encoding time of the visible light image of the current frame to be encoded, the fusion and encoding of the visible light image of the current frame to be encoded are delayed until the matching refresh time arrives.

[0019] In some examples, optionally, the field of view of the visible light imaging component is the same as the direction of the transmitted rays of the transmission scanning component; in the visible light image at the imaging time within the fusion time window, frame-by-frame fusion of the particulate features of the target object that have been transmitted and scanned by the transmission scanning component and are still within the channel segment includes: in the visible light image at the imaging time within the fusion time window, frame-by-frame fusion of a gradient mask layer generated based on the particulate features, wherein the particulate features are dynamically refreshed in response to the continuous movement of the target object, and the gradient mask layer is triggered to be generated in response to each fusion, such that the prominence of the particulate features is configured such that the particulate features characterize the coverage and occlusion of the particulate shape features of the same part.

[0020] In some examples, optionally, in the visible light images where the imaging time is within the fusion time window, a gradient mask layer generated based on the feature structure of the location is fused frame by frame, including: superimposing the gradient mask layer on a calibration image region of each frame of the visible light image where the imaging time is within the fusion time window, the calibration image region being the mapping region in the visible light image of the section between the scanning position and the exit side boundary of the section within the channel, and the layer size of the gradient mask layer matching the region size of the calibration image region.

[0021] In some examples, optionally, the transmission scanning component is configured to perform a line scan on the target object; the transmission scanning data includes line scan data generated for each line scan, which is used to generate a transmission ray strip image; the gradient mask layer includes the transmission ray strip images arranged and stitched in chronological order of scanning time; the transmission ray strip images generated based on the line scan data of each line scan are subjected to pseudo-color processing, which colorizes the pixels in the transmission ray strip images that characterize the structural features of the location, and sets the pixels in the transmission ray strip images that match the full-load background correction data of the channel segment to empty pixels with a transparent rendering effect.

[0022] In some examples, optionally, the refresh of the particulate feature is a queue refresh of a strip cache queue for storing the transmissive ray strip images, wherein: the queue refresh is triggered in response to the storage of the transmissive ray strip images caused by each line scan; the strip cache queue is configured as a first-in-first-out queue such that: the transmissive ray strip images in the strip cache queue are arranged in scan time order, and the queue position of the transmissive ray strip images in the strip cache queue is shifted once in response to each queue refresh in a queue exit direction matching the movement direction of the target object; the queue length of the strip cache queue is configured such that: the sum of the widths of a predetermined number of transmissive ray strip images matches a first layer size of the gradient mask layer in the layer width direction corresponding to the movement direction of the target object; the length of the transmissive ray strip images matches a second layer size of the gradient mask layer in the layer height direction; and the gradient mask layer generated in response to each fusion includes the transmissive ray strip images stored in the strip cache queue at that fusion.

[0023] Optionally, fusing the structural features of the target object in the image sequence of the visible light image further includes: in response to a signal state change of a switching control signal, fusing the overall structural features of the target object in the visible light image at an imaging time later than the fusion time window, wherein the fusion position of the overall structural features in the visible light image matches the image position of the overall shape features of the target object in the visible light image, so that the presentation state of the target object in the exit outside image region of the channel outside the exit of the transmission scanning channel in the over-test video switches from the overall shape features to the overall structural features; in response to a signal state reset of the switching control signal, stopping the fusion of the overall structural features, so that the presentation state of the target object in the exit outside image region switches back to the overall shape features.

[0024] In some examples, optionally, the structural features are subjected to intelligent feature recognition; in response to the anomaly recognition result generated by the intelligent feature recognition, a visual anomaly alarm tag is added frame by frame to the image sequence of the visible light image, wherein the visual anomaly alarm tag is matched with the position of the structural features or the shape features in the visible light image, so that the visual anomaly alarm tag moves synchronously in the over-inspection video in response to the continuous movement of the target object.

[0025] In another embodiment of this application, a video generation apparatus includes:

[0026] A video generation module is used to generate a visible light image using image acquisition data generated by a visible light imaging component. The visible light imaging component is configured to image the channel segment inside the transmission scanning channel and the channel segment outside the entrance and exit of the transmission scanning channel. The visible light image includes a continuous transmission segment in which the channel segment and the channel segment are connected to each other. Furthermore, an over-detection video generated based on the image sequence of the visible light image is used to present the movement process of the target object along the continuous transmission segment.

[0027] An image fusion module is used to fuse the structural features of the target object in an image sequence of a visible light image by using transmission scanning data generated by a transmission scanning component arranged inside the transmission scanning channel on the target object passing through a section of the channel. The structural features are matched with the position of the shape features of the target object included in the visible light image, so that the continuous movement process of the over-detection video presentation includes: a synchronous and gradual associated presentation process in which the structural features and the shape features change in response to the continuous movement of the target object.

[0028] In some examples, optionally, the visible light imaging component includes a first camera component, a second camera component, and a third camera component, wherein the imaging field of view of the first camera component covers the channel segment, and the imaging fields of view of the second and third camera components respectively cover the channel segment outside the entrance and exit of the transmission scanning channel; the image acquisition data generated by the visible light imaging component includes the channel image sequence generated by the first camera component and the channel image sequence generated by the second and third camera components; the video generation module is specifically configured to generate the visible light image by stitching together the synchronous frames in the channel image sequence and the channel image sequence.

[0029] In some examples, optionally, the image fusion module is specifically configured to: frame-by-frame fuse the structural features of the target object that have been transmissively scanned by the transmission scanning component and are still within the channel segment in the visible light image at the imaging time within the fusion time window, wherein the fusion time window is used to characterize the time range from the target object arriving at the scanning position of the transmission scanning component to the target object moving out of the channel segment, and the associated presentation process of the structural features and the shape features is configured to: during the period when any part of the target object becomes the scanned part of interest, the presentation state of the part in the over-inspection video switches from being highlighted by the corresponding part shape features to being highlighted by the part structural features, and the presentation state of the part switches back to being highlighted by the corresponding part shape features in response to the change in position of moving out of the channel segment.

[0030] In some examples, optionally, a time prediction module is also included, which determines the fusion time window based on the transmission speed of the transmission mechanism and the distance of the target object’s current position relative to the scanning position when the target object is detected in the channel segment outside the entrance of the transmission scanning channel.

[0031] In some examples, optionally, the image fusion module is further configured to: initiate the fusion of the location structural features when it is detected that the data content contained in the transmission scan data changes from the full-load background correction data of the road segment in the channel to the feature content of the location structural features.

[0032] In some examples, the image fusion module is optionally further configured to: stop the fusion of the structural features of the location when it is detected that the road segment within the channel becomes empty due to the removal of the target object.

[0033] In some examples, optionally, the particulate features are dynamically refreshed in response to continuous movement of the target object; the image fusion module is specifically configured to achieve pairing between the visible light image and the particulate features in the following manner: based on the time matching of the refresh time of the dynamic refresh with the imaging time of the visible light image, the dynamically refreshed particulate features are fused into the visible light image, so that the fused particulate features in the visible light image are refreshed at a refresh time that matches the imaging time of the visible light image.

[0034] In some examples, optionally, the over-detected video is generated by encoding the image sequence of the visible light image frame by frame; the image fusion module is specifically configured to implement the time matching in the following manner: in response to the arrival of the encoding time of each frame of the visible light image, a matching refresh time that matches the imaging time of the visible light image of the current frame to be encoded is determined; the part construction features that have been refreshed at the matching refresh time are fused into the visible light image of the current frame to be encoded.

[0035] In some examples, optionally, the site construction features are dynamically refreshed at the scanning frequency of the transmission scanning component, and the frame rate of the visible light image sequence is different from the scanning frequency; the image fusion module is specifically configured to determine the matching refresh time in the following manner, including: in a refresh time sequence matching the scanning frequency, determining the refresh time closest to the imaging time of the visible light image of the current frame to be encoded as the matching refresh time, wherein if the matching refresh time is later than the encoding time of the visible light image of the current frame to be encoded, the fusion and encoding of the visible light image of the current frame to be encoded are delayed until the matching refresh time arrives.

[0036] In some examples, optionally, the field of view of the visible light imaging component is the same as the direction of the transmitted rays of the transmission scanning component; the image fusion module is specifically configured to perform fusion in the following manner: in the visible light image located within the fusion time window at the imaging time, a gradient mask layer generated based on the part construction features is fused frame by frame, wherein the part construction features are dynamically refreshed in response to the continuous movement of the target object, and the gradient mask layer is triggered to be generated in response to each fusion, such that the prominence of the part construction features is configured such that the part construction features represent the coverage and occlusion of the part shape features of the same part.

[0037] In some examples, the image fusion module is optionally configured to overlay the gradient mask layer in the following manner: overlaying the gradient mask layer onto a calibration image region of each frame of the visible light image located within the fusion time window at the imaging time, wherein the calibration image region is the mapping region of the road segment interval between the scanning position and the exit side boundary of the road segment in the visible light image, and the layer size specification of the gradient mask layer matches the region size specification of the calibration image region.

[0038] In some examples, optionally, the transmission scanning component is configured to perform a line scan on the target object; the transmission scanning data includes line scan data generated for each line scan, which is used to generate a transmission ray strip image; the gradient mask layer includes the transmission ray strip images arranged and stitched in chronological order of scanning time; the transmission ray strip images generated based on the line scan data of each line scan are subjected to pseudo-color processing, which colorizes the pixels in the transmission ray strip images that characterize the structural features of the location, and sets the pixels in the transmission ray strip images that match the full-load background correction data of the channel segment to empty pixels with a transparent rendering effect.

[0039] Optionally, in some examples, a data caching module is also included, wherein a strip cache queue for storing the transmissive ray strip images is deployed in the data caching module, and the refresh of the part construction feature is a queue refresh of the strip cache queue, wherein: the queue refresh is triggered in response to the storage of the transmissive ray strip images caused by each line scan; the strip cache queue is configured as a first-in-first-out queue such that: the transmissive ray strip images in the strip cache queue are arranged in scan time order, and the queue position of the transmissive ray strip images in the strip cache queue is shifted once in response to each queue refresh in a queue exit direction matching the movement direction of the target object; the queue length of the strip cache queue is configured such that: the sum of the widths of a predetermined number of transmissive ray strip images matches the first layer size of the gradient mask layer in the layer width direction corresponding to the movement direction of the target object; the length of the transmissive ray strip images... Match the second layer size of the gradient mask layer in the layer height direction; in response to each fusion, the gradient mask layer generated includes the transmitted ray strip image stored in the strip cache queue during that fusion.

[0040] Optionally, in some examples, a signal monitoring module is also included for monitoring the signal state of the switching control signal; the image fusion module is further configured to: in response to a change in the signal state of the switching control signal, fuse the overall structural features of the target object in the visible light image at an imaging time later than the fusion time window, wherein the fusion position of the overall structural features in the visible light image matches the image position of the overall shape features of the target object in the visible light image, so that the presentation state of the target object in the out-of-channel segment corresponding to the exit of the transmission scanning channel in the over-test video switches from the overall shape features to the overall structural features, so that the presentation state of the target object in the out-of-channel segment corresponding to the exit of the transmission scanning channel in the over-test video switches from the overall shape features to the overall structural features; in response to a reset of the signal state of the switching control signal, stop the fusion of the overall structural features, so that the presentation state of the target object in the out-of-channel segment switches back to the overall shape features.

[0041] Optionally, in some examples, an intelligent recognition module is also included for intelligently recognizing the structural features, and, in response to the abnormal recognition result generated by the intelligent recognition of features, adding a visual abnormality alarm tag frame by frame in the image sequence of the visible light image, wherein the visual abnormality alarm tag is matched with the position of the structural features or the shape features in the visible light image, so that the visual abnormality alarm tag moves synchronously in the over-inspection video in response to the continuous movement of the target object.

[0042] In some examples, the intelligent recognition module is optionally configured to add the visual anomaly alarm tag frame by frame in the visible light image after the time when the imaging time is after the time when the anomaly recognition result is generated, so that the movement range of the visual anomaly alarm tag in response to the continuous movement of the target object covers the in-channel image area corresponding to the in-channel segment in the over-inspection video, and the out-of-channel image area corresponding to the out-of-channel segment outside the exit of the transmission scanning channel.

[0043] In another embodiment of this application, a detection system includes:

[0044] Transmission scanning channel;

[0045] A conveying mechanism for conveying the target object through the transmission scanning channel;

[0046] Visible light imaging component for imaging the channel section inside the transmission scanning channel and the channel section outside the entrance and exit of the transmission scanning channel.

[0047] A transmission scanning component is used to perform transmission scanning on the target object passing through the road segment within the channel;

[0048] A processing module configured to perform the video generation method described in the foregoing embodiments;

[0049] A communication component is used to establish a communication connection between the processing module and the video playback device, so that the passed-in video is transmitted to the video playback device for playback.

[0050] In another embodiment of this application, a video playback device includes:

[0051] Display module;

[0052] A processing module is configured to execute the video generation method described in the foregoing embodiments, and to play and present the passed-in video through the display module;

[0053] A communication component is provided to establish a communication connection between the processing module and a detection system including the transmission scanning channel, the transmission mechanism, the visible light imaging component, and the transmission scanning component. The communication connection is used by the processing module to acquire the image acquisition data and the transmission scanning data.

[0054] In another embodiment of this application, a non-transitory computer-readable storage medium is provided, wherein instructions are stored that, when executed by a processor, cause the processor to perform the video generation method described in the foregoing embodiments.

[0055] Based on the above embodiments, a visible light image that fully presents the continuous transmission path inside the transmission scanning channel can be obtained. Furthermore, by performing a transmission scan on the target object within the transmission scanning channel, the structural features of the target object can be fused into the image sequence of the visible light image. This allows the inspection video generated based on the image sequence of the visible light image to display the continuous, uninterrupted movement of the target object along the continuous transmission path in a way that correlates the structural and shape features of the target object. Thus, the correlation presentation of the structural and shape features of the target object in the inspection video helps inspectors improve the object matching efficiency of identifying the target object among physical objects. Attached Figure Description

[0056] The following figures are for illustrative purposes only and do not limit the scope of this application:

[0057] Figure 1 This is an exemplary structural diagram of the detection system in one embodiment of this application;

[0058] Figure 2 For example Figure 1 A schematic diagram of the image region distribution of the visible light image generated by the detection system in the illustrated embodiment;

[0059] Figure 3 For example Figure 1 A schematic diagram illustrating the principle of the continuous movement process of the target object presented in the over-detection video generated by the detection system in the illustrated embodiment;

[0060] Figure 4 For example Figure 1 The over-detection video generated by the detection system in the illustrated embodiment is a schematic diagram of the principle of the continuous movement process of the target object presented in a way that associates construction features with shape features;

[0061] Figure 5 For example Figure 1 A schematic diagram of the line scanning mode of the transmission scanning component of the detection system in the illustrated embodiment;

[0062] Figure 6 For example Figure 1 A schematic diagram illustrating the principle of the time-matching example used in the detection system of the illustrated embodiment;

[0063] Figure 7 For example Figure 1 A schematic diagram illustrating the principle of the transmission X-ray strip image management mechanism used in the detection system of the illustrated embodiment;

[0064] Figure 8 For example Figure 1 A schematic diagram illustrating the principle of the manual switching presentation mechanism used in the detection system of the illustrated embodiment;

[0065] Figure 9 For example Figure 1 A schematic diagram illustrating the principle of the intelligent recognition mechanism used in the detection system of the illustrated embodiment;

[0066] Figure 10 This is an exemplary structural diagram of a video playback device in another embodiment of this application;

[0067] Figure 11 This is an exemplary flowchart of a video generation method in another embodiment of this application;

[0068] Figure 12 For example Figure 11 The video generation method in the illustrated embodiment further incorporates an extended process diagram of a manual switching presentation mechanism;

[0069] Figure 13 For example Figure 11The video generation method in the illustrated embodiment further incorporates an extended process diagram of an intelligent recognition mechanism;

[0070] Figure 14 This is an exemplary flowchart of a video generation method in another embodiment of this application;

[0071] Figure 15 For example Figure 14 The video generation device in the illustrated embodiment further supports an extended structure diagram of a manual switching presentation mechanism;

[0072] Figure 16 For example Figure 14 The video generation device in the illustrated embodiment further supports an extended structure diagram of an intelligent recognition mechanism. Detailed Implementation

[0073] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided with reference to the accompanying drawings and embodiments.

[0074] Figure 1 This is an exemplary structural diagram of a detection system according to one embodiment of this application. Please refer to... Figure 1 In one embodiment of this application, the detection system based on transmission scanning imaging may include a transmission mechanism 10, a visible light imaging component 20, a transmission scanning channel 40, a processing module 50, and a communication module 60.

[0075] The conveying mechanism 10 is used to convey the target object through the transmission scanning channel 40, and the conveying mechanism 10 can be controlled by the processing module 50.

[0076] For example, the conveying mechanism 10 may include a conveyor belt 11 deployed through the transmission scanning channel 40 and a drive roller 12 for driving the conveyor belt 10, which may be controlled by the processing module 50. The transmission scanning channel 40 may include a shield (e.g., a rigid leaded shell) with openings at both ends, the openings at both ends being the entrance and exit of the transmission scanning channel 40, respectively. Shielding curtains (e.g., flexible leaded curtains) may be hung at the entrance and exit of the transmission scanning channel 40, respectively. The conveying direction of the conveying mechanism 10 (i.e., the conveyor belt 10) may be configured to enter the channel of the transmission scanning channel 40 from the entrance and exit from the exit of the transmission scanning channel 40. Target objects carried by the conveying mechanism 10 (i.e., the transmission belt 12) can pass through the entrance and exit of the transmission scanning channel 40 by compressing the shielding curtains.

[0077] The visible light imaging component 20 is used to image the channel segment S_scan located inside the transmission scanning channel 40 of the transmission mechanism 10, and the channel segments S_in and S_out located outside the entrance and exit of the transmission scanning channel 40. Furthermore, the visible light imaging component 20 can also be controlled by the processing module 50 and provide the processing module 50 with image acquisition data obtained from imaging the channel segment S_scan and the channel segments S_in and S_out.

[0078] For example, the visible light imaging assembly 20 includes a first camera assembly 21, a second camera assembly 22, and a third camera assembly 23. Each of these three camera assemblies can include a visible light-sensitive device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The imaging field of view of the first camera assembly 21 can cover the in-channel segment S_scan, while the imaging fields of view of the second and third camera assemblies 22 and 23 respectively cover the out-of-channel segment S_in outside the entrance and the out-of-channel segment S_out outside the exit of the transmission scanning channel 40. Accordingly, the image acquisition data generated by the visible light imaging assembly 20 can include the in-channel image sequence generated by the first camera assembly 21 and the out-of-channel image sequence generated by the second and third camera assemblies 22 and 23.

[0079] In addition, in the embodiments of this application, the field of view direction of the visible light imaging component 20 (i.e., the field of view direction of the first camera component 21, the second camera component 22 and the third camera component 23) is taken as a top view direction in the vertical direction perpendicular to the horizontal bearing surface of the conveying mechanism 10 (i.e., the upper surface of the conveyor belt 11). However, it is understood that the field of view direction of the visible light imaging component 20 can also be set as a bottom view direction in the vertical direction or a side view direction in the horizontal direction parallel to the horizontal bearing surface of the conveying mechanism 10.

[0080] The processing module 50 may include at least one of the following processing devices: CPU (central processing unit), logic device such as FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit), and AI (Artificial Intelligence) processing unit.

[0081] The processing module 50 can be used to generate a visible light image using the image acquisition data generated by the visible light imaging component 20.

[0082] Figure 2 For example Figure 1 A schematic diagram of the image region distribution of the visible light image generated by the detection system in the illustrated embodiment. (See diagram for example.) Figure 2 As shown, since the visible light imaging component 20 is configured to image the in-channel segment S_scan located inside the transmission scanning channel 40 of the transmission mechanism 10, and the out-channel segments S_in and S_out located outside the entrance and exit of the transmission scanning channel 40, the visible light image can include continuous transmission segments in which the in-channel segment S_scan and the out-channel segments S_in and S_out are connected to each other.

[0083] For example, when the visible light imaging component 20 includes the aforementioned first camera component 21, second camera component 22 and third camera component 23, the process of the processing module 50 generating a visible light image can be specifically configured as follows: a visible light image is generated by stitching together the synchronous frames in the in-channel image sequence generated by the first camera component 21 and the out-of-channel image sequence generated by the second camera component 22 and the third camera component 23. Furthermore, if there is a positional deviation between the imaging fields of the first camera assembly 21, the second camera assembly 22, and the third camera assembly 23, and / or, there is an angular deviation between the field of view directions of the first camera assembly 21, the second camera assembly 22, and the third camera assembly 23, and / or, the size specifications of the imaging fields of the first camera assembly 21, the second camera assembly 22, and the third camera assembly 23 differ from the image specifications of the visible light image, then the process of the processing module 50 generating the visible light image may further include: distortion correction processing of the synchronous frames in the image sequence within the channel and the image sequence outside the channel before stitching, and / or, image cropping processing of the synchronous frames in the image sequence within the channel and the image sequence outside the channel before stitching, and / or, de-overlap processing of the stitching seam area in the visible light image after stitching.

[0084] exist Figure 2 In the image, the corresponding image area of ​​the channel segment S_scan in the visible light image is represented by a shadow. This is because the channel segment S_scan is located inside the transmission scanning channel 40, and the ambient brightness inside the transmission scanning channel 40 is usually lower than the ambient brightness of the external environment where the channel segments S_in and S_out are located. In actual deployment, supplementary lights can be further installed inside the transmission scanning channel 40 to improve the ambient brightness within the imaging field of view of the first camera component 21.

[0085] The processing module 50 can also be used to generate over-detected video based on image sequences of visible light images. Furthermore, the communication component 60 is used to establish a communication connection between the processing module 50 and the video playback device 80. For example, the communication module 60 can be configured to support wireless communication such as Wi-Fi (Wireless Fidelity) and / or wired communication based on wired networks such as Ethernet or fiber optic networks. Accordingly, the processing module 50 can transmit the over-detected video to the video playback device 80 for playback and presentation through the communication module 60.

[0086] Figure 3 For example Figure 1 The schematic diagram illustrates the principle of the continuous movement process of the target object presented in the over-detection video generated by the detection system in the illustrated embodiment. Please refer to... Figure 3 Since the image position of the target object 70 in each frame of visible light image is associated with the transmission position when the target object 70 is imaged, in response to the continuous transmission of the target object 70 by the transmission mechanism, the over-detection video generated based on the image sequence of the visible light image can be used to present the movement process of the target object 70 along the continuous transmission path where the inner channel segment S_scan and the outer channel segments S_in and S_out are connected to each other.

[0087] That is, based on the embodiments of this application, the detection system can obtain a visible light image that fully presents the continuous transmission path inside the channel of the transmission scanning channel 40, and the over-detection video generated based on the image sequence of the visible light image can present the continuous movement process of the target object 70 along the continuous transmission path without any breaks (i.e. without any breaks inside the channel of the transmission scanning channel 40).

[0088] Please review Figure 1 In embodiments of this application, the detection system further includes a transmission imaging component 30 disposed inside the transmission scanning channel 40.

[0089] The transmission scanning component 30 is used to perform transmission scanning of a target object passing through a segment S_scan (i.e., the interior of the transmission scanning channel 40) using transmitted rays (e.g., X-rays). The transmission scanning component 30 can be controlled by the processing module 50 and can provide transmission scanning data to the processing module 50. The scanning position P_scan of the transmission scanning component 30 within the segment S_scan (i.e., the interior of the transmission scanning channel 40) can be located at the midpoint of the segment S_scan, or at any position within the segment S_scan closer to the entrance. Although in Figure 1The illustration uses the example of the scanning position P_scan being the same as the installation position of the transmission scanning component 30. However, it is understood that the scanning position P_scan is not the same as the installation position of the transmission scanning component 30. Furthermore, the scanning position P_scan can refer to the projection position of the coverage area of ​​the transmission rays generated by the transmission scanning component 30 in the channel segment S_scan. For example, the scanning position P_scan can be the boundary position of the projection area of ​​the transmission rays generated by the transmission scanning component 30 in the channel segment S_scan near the entrance side. In this case, the scanning position P_scan can be considered as the starting position of the target object entering the coverage area of ​​the transmission rays and starting the scanning.

[0090] For example, the transmission scanning component 30 may include a radiation source 31 for generating transmitted rays and a sensing array 32 for generating transmission scanning data by sensing the transmitted rays. If the target object is located at the scanning position P_scan of the transmission scanning component 30 within the channel segment S_scan (i.e., inside the channel of the transmission scanning channel 40), the transmitted rays generated by the radiation source 31 penetrate the target object and are projected onto the sensing array 32. The transmission scanning data generated by the sensing array 32 includes pixel values ​​corresponding to each array pixel. The pixel value of each array pixel characterizes the energy value of the transmitted rays sensed at that position after attenuation due to penetration of the target object. Since the degree of attenuation of the transmitted rays at different positions is related to the internal structure of the target object, the pixel values ​​corresponding to each array pixel can characterize the structural features of the target object. Furthermore, the radiation emission spectrum and / or radiation intensity generated by the radiation source 31 can be determined by the radiation characteristic parameters configured by the processing module 50. The imaging sensing parameters used by the sensing array 32 when converting the energy of the sensed transmitted rays into pixel values ​​can be configured by the processing module 50.

[0091] In addition, in the embodiments of this application, the direction of the transmitted rays of the transmission scanning component 30 is preferably configured to match the field of view of the visible light imaging component 20. For example, the ray source 31 and the sensing array 32 of the projection scanning component 30 are arranged opposite each other in the vertical direction perpendicular to the horizontal bearing surface (i.e. the upper surface of the conveyor belt 11) of the conveying mechanism 10, so that the direction of the transmitted rays of the transmission scanning component 30 and the field of view of the visible light imaging component 20 are both vertical.

[0092] The processing module 50 can also be used to utilize the transmission scanning component 30 arranged inside the transmission scanning channel 40 to generate transmission scanning data of the target object passing through the channel segment S_scan (i.e. passing through the channel inside the transmission scanning channel 40), and fuse the structural features of the target object in the image sequence of the visible light image, wherein the structural features of the target object are matched with the position of the shape features of the target object included in the visible light image, so that the continuous movement process of the over-detection video presentation includes: a correlation presentation process in which the structural features and shape features of the same target object change synchronously and gradually in response to the continuous movement of the target object.

[0093] In other words, based on the embodiments of this application, the detection system can perform a transmission scan of the target object within the transmission scanning channel 40, and can also fuse the structural features of the target object in the image sequence of the visible light image. This allows the pass-through video generated based on the image sequence of the visible light image to display the continuous movement of the target object along a continuous transmission path without interruption, in a way that presents the structural features and shape features of the same target object in a way that is associated with each other. Thus, the associated presentation of the structural features and shape features of the target object in the pass-through video helps the inspector improve the object matching efficiency of identifying the target object among physical objects.

[0094] In the embodiments of this application, while enhancing the correlation between the structural features and shape features of the same target object by using position matching of the structural features and shape features, in order to take into account the intuitive visibility of the structural features and shape features respectively, it is preferable to use a method in which the structural features and shape features are presented alternately along the movement trajectory of the target object.

[0095] Accordingly, the processing module 50 can fuse frame by frame the structural features of the target object that has been transmissively scanned by the transmission scanning component and is still in the channel segment S_scan in the visible light image located within the fusion time window at the imaging time. The fusion time window is used to characterize the time range from when the target object arrives at the scanning position P_scan of the transmission scanning component 30 to when the target object moves out of the channel segment S_scan.

[0096] For example, when a target object is detected in the outer segment S_in outside the entrance of the transmission scanning channel 40, the processing module 50 can determine a fusion time window based on the transmission speed of the conveying mechanism 10 and the distance between the current position of the target object and the scanning position S_scan. The process of determining the fusion time window can be viewed as a time prediction process. To detect the target object in the outer segment S_in outside the entrance of the transmission scanning channel 40, a photoelectric sensor deployed in the outer segment S_in can be used. A level change in the detection signal generated by the photoelectric sensor can characterize the arrival of the target object in the outer segment S_in. The processing module 50 can determine whether a newly arrived target object is in the outer segment S_in outside the entrance of the transmission scanning channel 40 based on the level state of the detection signal generated by the photoelectric sensor. Alternatively, the processing module 50 can also determine whether a newly arrived target object is in the outer segment S_in outside the entrance of the transmission scanning channel 40 by performing target recognition on a visible light image.

[0097] Figure 4 For example Figure 1 The over-detection video generated by the detection system in the illustrated embodiment is a schematic diagram illustrating the principle of the continuous movement process of the target object presented in a way that associates construction features with shape features. Please refer to [link to relevant documentation]. Figure 4 In embodiments of this application, the process of associating the structural features and shape features of the target object in the over-inspection video can be configured as follows:

[0098] During the period when any part of the target object 70 becomes a scanned area of ​​interest, the presentation state of that part in the over-checked video changes from being highlighted by the corresponding part's shape features (in... Figure 4 The external image area of ​​the passage segment S_in corresponding to the entrance side is represented as a blank area without shadows, and then switched to a representation highlighting the structural features of the aforementioned location (in...). Figure 4 The corresponding image area within the channel of the road segment S_scan is represented by a shadow, and the presentation state of this part responds to the change in position of the road segment moving out of the channel, switching back to a representation highlighted by the corresponding part's shape features (in...). Figure 4 The outer channel area of ​​the corresponding exit side channel segment S_out is represented as a blank area without shadows.

[0099] In other words, during the continuous movement of the target object, the parts of the target object located outside the channel S_in and S_out are presented in the over-inspection video in a way that is visually visible in terms of shape features, while the scanned parts of interest of the target object still inside the channel S_scan are presented in the over-inspection video in a way that is visually visible in terms of structural features. That is, visually, it is similar to adding a through-filter to the image area inside the channel corresponding to the channel S_scan in the over-inspection video.

[0100] During the continuous movement of the target object, the scanned areas of interest of the target object shift in response to the continuous movement. Correspondingly, the feature content and position of the shape features of the scanned areas of interest in the visible light image, as well as the feature content and position of the structural features fused into the visible light image, will dynamically change with the shift of the scanned areas of interest of the target object. Therefore, in order to ensure that the positional matching of the structural features and shape features of the scanned areas of interest of the same target object is achieved, the structural features fused into the visible light image can be dynamically refreshed in response to the continuous movement of the target object. Furthermore, the processing module 50 can be specifically configured as follows:

[0101] Based on the time matching of the dynamic refresh time of the regional structural features with the imaging time of the visible light image, the dynamically refreshed regional structural features are fused into the visible light image, so that the fused regional structural features in the visible light image are refreshed at the refresh time that matches the imaging time of the visible light image.

[0102] The imaging time of the visible light image can be the generation time of the visible light image. However, in order to eliminate the delay caused by the generation process of the visible light image, the imaging time of the visible light image can also be set to the acquisition time of the synchronization frame used to generate the visible light image in the image sequence within the channel and the image sequence outside the channel. Alternatively, the imaging time of the visible light image can be set to the compensation result of the generation time of the visible light image using a preset compensation value.

[0103] In the embodiments of this application, the over-detection video can be generated by encoding the image sequence of the visible light image frame by frame. Therefore, the processing module 50 can achieve time matching between the refresh time of the dynamic refresh of the part structure features and the imaging time of the visible light image in the following manner:

[0104] In response to the arrival of the encoding time of each frame of visible light image, a matching refresh time that matches the imaging time of the visible light image of the current frame to be encoded is determined;

[0105] The structural features of the parts that will be refreshed at the determined matching refresh time will be fused into the visible light image of the current frame to be encoded.

[0106] In embodiments of this application, the transmission scanning component 30 can be configured to perform a line scan on the target object.

[0107] Figure 5 For example Figure 1 A schematic diagram of the line scanning mode of the transmission scanning component of the detection system in the illustrated embodiment. Please refer to [link / reference]. Figure 5The X-ray source 31 of the transmission scanning assembly 30 can be shielded by a slit-like shield (e.g., a lead block) so that only a portion of the scattered rays generated by the X-ray source 31 can pass through the slits of the shield and reach the induction array 32. Accordingly, the induction array 32 can be configured as a linear array with dimensions corresponding to the length and width of the slits. Thus, a transmission X-ray radiation surface 300 can be formed at the scanning position P_scan in the channel segment S_scan, extending between the slits of the shield and the induction array 32. The coverage area of ​​the transmission X-ray radiation surface 300 in the transmission direction is small, i.e., the scan ends as soon as it begins. Therefore, in the case of using a line scan method, the scanning position P_scan can be considered as the line projection position of the transmission X-ray radiation surface 300 in the channel segment S_scan.

[0108] In this case, the refresh time of the dynamic refresh of the part structure features is distributed according to the scanning frequency of the line scan, while the imaging time of the visible light image is distributed according to the frame rate of the visible light image.

[0109] Figure 6 For example Figure 1 The schematic diagram illustrates the principle of the time-matching example used in the detection system of the illustrated embodiment. Please refer to [link / reference]. Figure 6 The structural features of a part can be dynamically refreshed at the scanning frequency of the transmission scanning component (i.e., the scanning frequency of the line scan) (the structural features of a part are refreshed once every 10~100ms). The frame rate of the visible light image sequence is different from the scanning frequency of the transmission scanning component. Optionally, the frame rate of the visible light image sequence can be lower than the scanning frequency of the transmission scanning component. In this case, for the image sequence {I_vis} of the visible light image whose imaging time is within the fusion time window T_merge, the processing module 50 can select the refresh time (I_vis) that is closest to the imaging time t_cap of the visible light image of the current frame to be encoded from the refresh time sequence {t_ref} that matches the scanning frequency. Figure 6 In the refresh time sequence {t_ref}, the dots on the coordinate axis represent the refresh times included in the refresh time sequence {t_ref}. The matching refresh time is determined to be the one that matches the imaging time of the visible light image of the current frame to be encoded. If the determined matching refresh time is later than the encoding time of the visible light image of the current frame to be encoded, the fusion and encoding of the visible light image of the current frame to be encoded can be delayed until the determined matching refresh time arrives.

[0110] In order to further enhance the intuitive visibility of structural features and shape features, and to minimize the time consumed by the fusion processing of visible light images, in the embodiments of this application, the processing module 50 can fuse the gradient mask layer generated based on the part structural features frame by frame in the visible light image that is within the fusion time window at the imaging time.

[0111] The part construction features are dynamically refreshed in response to the continuous movement of the target object, and the gradient mask layer is triggered to be generated in response to each fusion, so as to ensure that each superimposed gradient mask layer is generated by the part construction features that are refreshed at the matching refresh time, thereby making the part construction feature highlighting configured as follows: the part construction features represent the coverage and occlusion of the part shape features of the same part.

[0112] For example, if the transmission scanning component 30 is configured to perform line scanning on a target object, the transmission scanning data generated by the transmission scanning component 30 may include line scanning data generated for each line scan. This line scanning data can be used to generate a transmission ray strip image. Furthermore, the gradient mask layer may include transmission ray strip images stitched together in chronological order of scanning time, wherein:

[0113] The transmitted ray strip image generated based on the line scan data of each line scan is processed with pseudo-color to colorize the pixels of the feature content used to characterize the structural features of the part in the transmitted ray strip image. In addition, the pixels in the transmitted ray strip image that match the full-load background correction data of the road segment in the channel are set to empty pixels with a transparent presentation effect.

[0114] Among them, pseudocoloring refers to the process of assigning color values ​​to grayscale pixels. Since the pixel information of the feature content representing the structural features of the part in the line scan data can only be presented as a grayscale image, the pseudocolor image obtained by pseudocoloring can make the transmitted ray strip image contain richer color information, so as to improve the recognition of the structural features of the part after being fused into the visible light image by the superposition of the gradient mask layer. Full load background correction refers to the measurement data of the transmission scanning component 30 in the channel section S_scan when it is in an unloaded state. Full load local correction data includes full load data and background data. Full load data is the measurement data sensed by the sensing array 32 when the transmitted ray emission energy of the ray source 31 is fully emitted. Background data is the measurement data sensed by the sensing array 32 when the ray source 31 has no transmitted ray emission. Both can represent the background area (e.g., the surface features of the conveyor belt 11) in the channel section S_scan where there is no target object 70.

[0115] In this case, all areas in the gradient mask layer except for the layer area representing the structural features of the part can be transparent. Furthermore, the gradient mask layer becomes globally transparent when the channel segment S_scan is in an unloaded state. To ensure that the gradient mask layer is overlaid onto the visible light image when it contains structural features, and to simultaneously prevent the gradient mask layer from being overlaid onto the visible light image even when it is globally transparent, the processing module 50 can be further configured to:

[0116] When the data content of the transmission scan data generated by the transmission scan component 30 is detected to change from the full-load background correction data of the channel segment S_scan to the feature content of the part structure features of the target object, the processing module 50 can start the fusion of the part structure features in the visible light image.

[0117] When the processing module 50 detects that the road segment S_scan in the channel becomes empty due to the removal of the target object, the processing module 50 can stop the fusion of the part construction features in the visible light image.

[0118] In the embodiments of this application, the processing module 50 can overlay the current gradient mask layer onto the calibration image region of each frame of visible light image located within the fusion time window at each time the gradient mask layer is overlaid. That is, the layer content (e.g., part structural features) contained in the gradient mask layer can vary each time it is overlaid, but the overlay position of the gradient mask layer in the visible light image can be consistent each time. Therefore, the calculation process of the overlay position of the gradient mask layer in the visible light image can be eliminated, thereby improving the overlay efficiency of the gradient mask layer in the visible light image. The calibration image region can be the mapping region of the road segment interval between the scan position P_scan and the exit side boundary of the road segment S_scan in the channel in the visible light image, and the layer size of the gradient mask layer matches the region size of the calibration image region. It is understandable that the layer size of the gradient mask layer matches the area size of the calibration image region. This could mean that the layer size of the gradient mask layer is exactly the same as the area size of the calibration image region, or it could mean that the layer size of the gradient mask layer and the area size of the calibration image region meet a preset scaling ratio.

[0119] Since the generation of the gradient mask layer is triggered by fusion (e.g., triggered by the arrival of the encoding time of each frame of visible light image), while the refresh of the part construction features is triggered by scanning (e.g., line scanning), the triggering frequencies of the two are different. Therefore, in order to facilitate decoupling between fusion and scanning, in the embodiments of this application, a strip buffer queue can be used to manage the transmitted X-ray strip image. This strip buffer queue can be located in the storage module of the detection system.

[0120] Figure 7 For example Figure 1 The schematic diagram illustrates the principle of the transmission X-ray strip image management mechanism used in the detection system of the illustrated embodiment. Please refer to [link / reference needed]. Figure 7 In embodiments of this application, the refreshing of the part construction features can be achieved by refreshing the strip buffer queue Seq_strp used to store the transmitted ray strip image, wherein:

[0121] The strip buffer queue Seq_strp is refreshed in response to the storage of the transmitted ray strip image triggered by each line scan;

[0122] The strip buffer queue Seq_strp is configured as a first-in-first-out queue such that: the transmitted ray strip images in the strip buffer queue Seq_strp are arranged in scan time order, and the queue position of the transmitted ray strip image in the strip buffer queue Seq_strp is shifted once in the queue out direction that matches the movement direction of the target object in response to each queue refresh;

[0123] The queue length of the strip buffer queue Seq_strp is configured such that: the sum of the widths of a predetermined number of transmitted ray strip images matches the first layer size of the gradient mask layer I_mask in the layer width direction of the corresponding target object's movement direction, wherein the width of the transmitted ray strip image is associated with the line width dimension of the line scan of the transmission scanning component 30, the line width dimension of the line scan of the transmission scanning component 30 can be determined by the slit width dimension of the occluder of the ray source 31, the layer width direction of the gradient mask layer I_mask corresponds to the transmission direction of the transmission mechanism 10, and the first layer size of the gradient mask layer I_mask in the layer width direction can be matched with the distance between the scan position P_scan and the exit side boundary of the channel segment S_scan;

[0124] The length of the transmitted ray strip image stored in the strip buffer queue Seq_strp matches the second layer size of the gradient mask layer I_mask in the layer height direction. The layer height direction of the gradient mask layer I_mask corresponds to the transverse direction perpendicular to the conveying direction of the conveyor mechanism 10, and the second layer size of the gradient mask layer I_mask in the layer height direction can match the width of the conveyor belt 11.

[0125] Thus, the gradient mask layer I_mask generated in response to each fusion includes the transmitted ray strip image stored in the strip buffer queue Seq_strp at the time of that fusion.

[0126] The aforementioned method of fusing the structural features of a target object in an image sequence of a visible light image is an uncontrollable automatic fusion of the structural features of the scanned areas of interest of the target object in the visible light image. Based on this method, after the target object is completely removed from the channel segment S_scan, the target object can only be presented in its overall shape in the image area outside the channel segment S_out corresponding to the exit of the transmission scanning channel in the inspection video. If the inspector misses the presentation process of the structural features of the scanned areas of interest of the target object in the inspection video, he can rewatch the inspection video, but this will delay the object matching of the target object.

[0127] Therefore, in the embodiments of this application, fusing the structural features of the target object into the image sequence of a visible light image may also include another controllable fusion method based on manual switching, so that the inspection personnel can manually switch the screen for review.

[0128] Figure 8 For example Figure 1 The schematic diagram illustrates the principle of the manual switching presentation mechanism used in the detection system of the illustrated embodiment. Please refer to [link / reference]. Figure 8 The processing module 50 can be further used for:

[0129] In response to the signal state change of the switching control signal Sig_sw generated by the video playback device, the overall structural features of the target object 70 are fused in the visible light image where the imaging time is later than the fusion time window. The fusion position of the overall structural features of the target object 70 in the visible light image matches the image position of the overall shape features of the target object 70 in the visible light image. This ensures that the target object's presentation state in the external image area of ​​the external segment S_out of the transmission scanning channel corresponding to the exit of the over-examination video is determined from the overall shape features (in... Figure 8 The outer channel area of ​​the corresponding exit side of the channel S_out is represented as a blank area without shadows. This is then switched to the overall structural features (in...). Figure 8 The area outside the channel on the corresponding exit side of the channel, S_out, is represented by a shadow.

[0130] In response to the signal state reset of the switching control signal Sig_sw, the fusion of the overall structural features is stopped, so that the presentation state of the target object in the exit area of ​​the over-inspection video switches back to the overall shape features.

[0131] For example, the storage module of the detection system can be further equipped with a strip stack queue Seq_stck. The transmitted ray strip images generated based on the line scan data of each line scan (e.g., after pseudo-color processing) can be stored in the strip stack queue Seq_stck in the order of scan time. When the signal state of the switching control signal Sig_sw changes, the processing module 50 can obtain all the transmitted ray strip images stored in the strip stack queue Seq_stck and generate a temporary mask layer to characterize the overall structural features of the target object 70. The processing module 50 will also superimpose the generated temporary mask layer onto the visible light image whose imaging time is later than the fusion time window. Furthermore, the processing module 50 can determine the superposition position of the temporary mask layer in each frame of visible light image by identifying the image position of the overall shape features of the target object 70 in the visible light image, or by predicting the position of the target object 70 based on the transmission speed of the transmission mechanism 10, so that the superposition position of the temporary mask layer in each frame of visible light image matches the image position of the overall shape features of the target object 70 in that frame of visible light image.

[0132] In addition to fusing the structural features of the target object into the image sequence of the visible light image, in the embodiments of this application, the structural features of the target object can be intelligently identified to further present a visual abnormality alarm label that moves with the target object in the over-inspection video.

[0133] Figure 9 For example Figure 1 The schematic diagram illustrates the principle of the intelligent recognition mechanism used in the detection system of the illustrated embodiment. Please refer to [link / reference]. Figure 9 In embodiments of this application, the processing module 50 may be further used for:

[0134] Intelligent feature recognition is performed on the structural features of the target object 70 (e.g., the structural features of the scanned area of ​​interest).

[0135] In response to the anomaly recognition results generated by feature-based intelligent recognition, a visual anomaly alarm tag Tag_alr is added frame by frame to the image sequence of visible light images;

[0136] Among them, the visual anomaly alarm tag Tag_alr is matched with the position of the structural features or shape features in the visible light image so that the visual anomaly alarm tag Tag_alr moves synchronously in response to the continuous movement of the target object in the over-inspection video.

[0137] For example, feature intelligent recognition can be implemented based on a pre-trained neural network; and the processing module 50 can add a visual anomaly alarm tag Tag_alr frame by frame in all visible light images after the anomaly recognition result is generated at the imaging time, so that the visual anomaly alarm tag moves synchronously in response to the continuous movement of the target object, covering the intra-channel image area of ​​the corresponding intra-channel segment S_scan in the over-inspection video, and the out-of-channel image area of ​​the corresponding transmissive scanning channel 40 outer-channel segment S_out.

[0138] In the above embodiments of this application, the fusion of the structural features of the target object in the visible light image and the addition of the visual alarm tag in the visible light image are both completed by the detection system (i.e., implemented by the processing module 50 of the detection system). Furthermore, the video playback device can directly play the over-detected video generated by the detection system (i.e., the processing module 50) without further processing.

[0139] In another embodiment of this application, the fusion of the structural features of the target object in the visible light image and the addition of the visual alarm tag in the visible light image can be performed by the video playback device. In this case, the detection system can simply transmit the image acquisition data generated by the visible light imaging component 20 and the transmission scanning data generated by the transmission scanning component 30 to the video surveillance device.

[0140] Figure 10 This is an exemplary structural diagram of a video playback device according to another embodiment of this application. Please refer to... Figure 10 In another embodiment of this application, the video playback device may include a display module 1010, a processing module 1030, and a communication component 1050.

[0141] The processing module 1030 of the video playback device can be basically the same as the processing module 50 in the detection system in the aforementioned embodiment. The communication component 1050 of the video playback device is used to establish a communication connection between the processing module 1030 of the video playback device and the detection system including the transmission scanning channel 40, the transmission mechanism 10, the visible light imaging component 20 and the transmission scanning component 30. This communication connection is used for the processing module 1030 of the video playback device to acquire image acquisition data required to generate a visible light image, to fuse the structural features of the target object into the visible light image, and to add a visual alarm tag to the visible light image.

[0142] See also Figure 10The video playback device in this embodiment may further include a storage module 1070 for maintaining the stripe buffer queue Seq_strp and the stripe stack queue Seq_stck, and a human-machine interaction module 1090 for generating the switching control signal Sig_sw. The storage module 1070 may include at least one device containing a storage medium, such as RAM (Random Access Memory), FLASH, and a physical disk; the human-machine interaction module 1090 may include at least one component that can be used to configure the signal state of the switching control signal Sig_sw through human-machine interaction, such as a mouse, a keyboard, and a touch panel integrated into the display module 1010.

[0143] When the fusion of the structural features of the target object in the visible light image and the addition of the visual alarm tag in the visible light image are performed by the video playback device, the technical effect can be basically the same as that of the detection system in the aforementioned embodiments, which will not be described in detail here.

[0144] In another embodiment of this application, a video generation method adaptable to different execution entities is also provided.

[0145] Figure 11 This is an exemplary flowchart illustrating a video generation method in another embodiment of this application. Please refer to... Figure 11 In this embodiment, the video generation method may include:

[0146] S1110: Using image acquisition data generated by the visible light imaging component, a visible light image is generated, wherein the visible light imaging component is configured to image the channel segment inside the transmission scanning channel and the channel segment outside the entrance and exit of the transmission scanning channel. The visible light image includes a continuous transmission segment in which the channel segment and the channel segment are connected to each other. Furthermore, an over-detection video generated based on the image sequence of the visible light image is used to present the movement process of the target object along the continuous transmission segment.

[0147] For example, if the visible light imaging component includes a first camera component, a second camera component, and a third camera component, where the imaging field of view of the first camera component covers the channel segment, and the imaging fields of view of the second and third camera components respectively cover the channel segments outside the entrance and exit of the transmission scanning channel, then the image acquisition data generated by the visible light imaging component can include the channel image sequence generated by the first camera component and the channel image sequence generated by the second and third camera components. Furthermore, step S1110 can specifically include: stitching together synchronous frames from the channel image sequence and the channel image sequence to generate a visible light image. Additionally, to improve the image quality of the visible light image, step S1110 can further include: distortion correction processing of the synchronous frames in the channel image sequence and the channel image sequence before stitching, and / or, image cropping processing of the synchronous frames in the channel image sequence and the channel image sequence before stitching, and / or, de-overlap processing of the stitching seam area in the visible light image after stitching.

[0148] S1130: Using a transmission scanning component arranged inside the transmission scanning channel to generate transmission scanning data of a target object passing through a section of the channel, the structural features of the target object are fused into the image sequence of the visible light image, wherein the fused structural features are matched with the position of the shape features of the target object included in the visible light image, so that the continuous movement process of the over-detection video presentation includes: a correlation presentation process in which the structural features and shape features of the target object change synchronously and gradually in response to the continuous movement of the target object.

[0149] Based on the video generation method described above, a visible light image that fully presents the continuous transmission path inside the transmission scanning channel can be obtained. Furthermore, by performing a transmission scan of the target object within the transmission scanning channel, the structural features of the target object can be fused into the image sequence of the visible light image. This allows the pass-through video generated from the image sequence of the visible light image to showcase the continuous, uninterrupted movement of the target object along the continuous transmission path in a way that correlates the structural and shape features of the target object. Thus, the correlation presentation of the structural and shape features of the target object in the pass-through video helps inspectors improve the efficiency of object matching in identifying target objects among physical objects.

[0150] In order to enhance the correlation between the structural features and shape features of the same target object by using position matching of the structural features and shape features, while taking into account the intuitive visibility of the structural features and shape features respectively, S1130 preferably adopts a method of alternating presentation of structural features and shape features along the movement trajectory of the target object. That is, S1130 may specifically include: in the visible light image located within the fusion time window at the imaging time, fusing frame by frame the structural features of the scanned part of interest of the target object that has been transmissively scanned by the transmissive scanning component and is still in the channel segment. The fusion time window is used to characterize the time range from when the target object arrives at the scanning position of the transmissive scanning component to when the target object moves out of the channel segment.

[0151] For example, before S1130, the video generation method may further include: when a target object located on an external segment outside the entrance of the transmission scanning channel is detected, determining a fusion time window based on the transmission speed of the transmission mechanism and the distance between the current position of the target object and the scanning position. For the detection method of the target object located on an external segment outside the entrance of the transmission scanning channel, please refer to the aforementioned embodiments, which will not be repeated here.

[0152] Furthermore, based on the specific method described above in S1130, the process of associating the structural features and shape features of the target object in the over-inspection video can be configured as in the aforementioned embodiments. Figure 4 As shown, during the period when any part of the target object becomes a scanned part of interest, the presentation state of that part in the over-inspection video changes from being highlighted by the corresponding part's shape features to being highlighted by the part's structural features. Furthermore, the presentation state of that part switches back to being highlighted by the corresponding part's shape features in response to a change in the position of the segment that moves out of the channel.

[0153] To ensure positional matching between the scanned feature structure and shape of the same target object's region of interest, the feature structure fused into the visible light image can be dynamically refreshed in response to the continuous movement of the target object. Furthermore, the process of fusing the scanned feature structure of the target object's region of interest frame by frame in the visible light image within the fusion time window at the imaging time (S1130) can specifically include:

[0154] Based on the time matching of the refresh time of the dynamic refresh of the part structure features and the imaging time of the visible light image, the dynamically refreshed part structure features are fused into the visible light image, so that the fused part structure features in the visible light image are refreshed at the refresh time that matches the imaging time of the visible light image. The method for determining the imaging time of the visible light image can be referred to the aforementioned embodiments, and will not be repeated here.

[0155] For example, the over-detection video can be generated by encoding the image sequence of the visible light image frame by frame, and the time matching implemented in S1130 can include: in response to the arrival of the encoding time of each frame of the visible light image, determining a matching refresh time that matches the imaging time of the visible light image of the current frame to be encoded; and constructing features of the parts that are refreshed at the determined matching refresh time and fusing them into the visible light image of the current frame to be encoded.

[0156] When the transmission scanning component is configured to perform a line scan on the target object, S1130 can determine the matching refresh time using the methods described in the previous embodiments, such as... Figure 6 The method is as follows: In the refresh time sequence that matches the scanning frequency, the refresh time that is closest to the imaging time of the visible light image of the current frame to be encoded is determined as the matching refresh time that matches the imaging time of the visible light image of the current frame to be encoded. Since the particulate structural features are dynamically refreshed at the scanning frequency of the transmission scanning component (i.e., the scanning frequency of the line scan), the frame rate of the visible light image sequence is different from the scanning frequency. Therefore, the determined matching refresh time may be exactly the same as the imaging time of the visible light image of the current frame to be encoded. Furthermore, if the determined matching refresh time is later than the encoding time of the visible light image of the current frame to be encoded, the fusion and encoding of the visible light image of the current frame to be encoded can be delayed until the determined matching refresh time arrives.

[0157] To further enhance the intuitive visibility of structural features and shape features, and to minimize the time consumed in the fusion processing of visible light images, in the embodiments of this application, S1130 can fuse a gradient mask layer generated based on the part structural features frame by frame in the visible light image located within the fusion time window at the imaging time. The part structural features are dynamically refreshed in response to the continuous movement of the target object, and the gradient mask layer is triggered to be generated in response to each fusion, so as to ensure that each superimposed gradient mask layer is generated by the part structural features that are refreshed at the matching refresh time. Thus, the highlighting of part structural features is configured such that the part structural features represent the coverage and occlusion of the part shape features of the same part.

[0158] For example, if the transmission scanning component is configured to perform line scanning on a target object, the transmission scanning data generated by the transmission scanning component can include the line scanning data generated for each line scan. This line scanning data can be used to generate a transmission ray strip image, and the generated transmission ray strip image can be managed using a strip buffer queue, as detailed in the foregoing embodiments. Figure 7The explanation will not be repeated here; furthermore, the gradient mask layer may include transmission ray strip images arranged and stitched in the order of scanning time, wherein: the transmission ray strip images generated based on the line scan data of each line scan are processed with pseudo-color to colorize the pixels of the feature content used to characterize the structural features of the part in the transmission ray strip images, and the pixels in the transmission ray strip images that match the full-load background correction data of the road segment in the channel are set to empty pixels with transparent rendering effect.

[0159] In this case, in order to perform a fusion operation of overlaying the gradient mask layer onto the visible light image when it contains the structural features, and at the same time to avoid performing a fusion operation of overlaying the gradient mask layer onto the visible light image when it is globally transparent, S1130 may further include: when it is detected that the data content contained in the transmission scan data changes from the full-load background correction data of the road segment in the channel to the feature content of the structural features, the fusion of the structural features in the visible light image is initiated; and when it is detected that the road segment in the channel becomes empty due to the removal of the target object, the fusion of the structural features in the visible light image is stopped.

[0160] Furthermore, the method by which S1130 superimposes a gradient mask layer on a visible light image can be as follows: the gradient mask layer is superimposed on the calibration image region of each frame of visible light image located within the fusion time window at the imaging time. This calibration image region is the mapping region of the road segment interval between the scanning position and the exit side boundary of the road segment in the channel in the visible light image, and the layer size specification of the gradient mask layer matches the region size specification of the calibration image region.

[0161] Figure 12 For example Figure 11 The video generation method in the illustrated embodiment further incorporates an extended process diagram of a manual switching presentation mechanism. Please refer to [link to diagram]. Figure 12 The video generation method in this embodiment may further include the following steps after S1130:

[0162] S1151: In response to the signal state change of the switching control signal, the overall structural features of the target object are fused in the visible light image where the imaging time is later than the fusion time window. The fusion position of the overall structural features of the target object in the visible light image is matched with the image position of the overall shape features of the target object in the visible light image, so that the presentation state of the target object in the outside image area of ​​the outside segment of the transmission scanning channel corresponding to the exit of the over-inspection video is switched from the overall shape features to the overall structural features.

[0163] The specific implementation of S1151 can be referred to in the foregoing embodiments regarding... Figure 8 The description will not be repeated here.

[0164] S1153: In response to the signal state reset of the switching control signal, stop the fusion of the overall structural features so that the presentation state of the target object in the exit screen area of ​​the over-inspection video switches back to the overall shape features.

[0165] Figure 13 For example Figure 11 The video generation method in the illustrated embodiment further incorporates an extended process diagram of an intelligent recognition mechanism. Please refer to [link / reference]. Figure 13 The video generation method in this embodiment may further include the following steps no earlier than S1130:

[0166] S1171: Perform intelligent feature recognition on the structural features of the target object (e.g., the structural features of the scanned area of ​​interest).

[0167] S1173: In response to the anomaly recognition result generated by feature intelligent recognition, a visual anomaly alarm tag is added frame by frame in the image sequence of the visible light image. The visual anomaly alarm tag is matched with the position of the structural feature or shape feature in the visible light image so that the visual anomaly alarm tag moves synchronously in response to the continuous movement of the target object in the over-inspection video.

[0168] The specific implementation methods of S1171 and S1173 can be referred to in the foregoing embodiments regarding... Figure 9 The description will not be repeated here.

[0169] In addition, such as Figure 12 He Ru Figure 13 The two extended processes can be combined, and the combined process will not be described in detail here.

[0170] In another embodiment of this application, a video generation apparatus that can be implemented by a computer program and carried out by different execution entities is also provided.

[0171] Figure 14 This is an exemplary structural diagram of a video generation apparatus according to another embodiment of this application. Please refer to... Figure 14 In this embodiment, the video generation device may include a video generation module 1410 and an image fusion module 1430.

[0172] The video generation module 1410 is used to generate a visible light image using image acquisition data generated by the visible light imaging component. The visible light imaging component is configured to image the channel segments inside the transmission scanning channel and the channel segments outside the entrance and exit of the transmission scanning channel. The visible light image includes continuous transmission segments in which the channel segments and the channel segments are connected to each other. Furthermore, an over-detection video generated based on the image sequence of the visible light image is used to present the movement process of the target object along the continuous transmission segments.

[0173] For example, if the visible light imaging component includes a first camera component, a second camera component, and a third camera component, where the imaging field of view of the first camera component covers the channel segment, and the imaging fields of view of the second and third camera components respectively cover the channel segments outside the entrance and exit of the transmission scanning channel, then the image acquisition data generated by the visible light imaging component can include the channel image sequence generated by the first camera component and the channel image sequence generated by the second and third camera components. Furthermore, the video generation module 1410 can generate a visible light image by stitching together the synchronous frames in the channel image sequence and the channel image sequence. In addition, to improve the image quality of the visible light image, the video generation module 1410 can further perform distortion correction processing on the synchronous frames in the channel image sequence and the channel image sequence before stitching, and / or perform image cropping processing on the synchronous frames in the channel image sequence and the channel image sequence before stitching, and / or perform de-overlap processing on the stitching seam area in the visible light image after stitching.

[0174] The image fusion module 1430 is used to fuse the structural features of the target object in the image sequence of the visible light image by using the transmission scanning data generated by the transmission scanning component arranged inside the transmission scanning channel for the target object passing through the channel. The fused structural features are matched with the position of the shape features of the target object included in the visible light image, so that the continuous movement process of the over-detection video presentation includes: a correlation presentation process in which the structural features and shape features of the target object change synchronously and gradually in response to the continuous movement of the target object.

[0175] Based on the aforementioned video generation device, a visible light image that fully presents the continuous transmission path inside the transmission scanning channel can be obtained. Furthermore, by performing a transmission scan of the target object within the transmission scanning channel, the structural features of the target object can be fused into the image sequence of the visible light image. This allows the inspection video generated based on the image sequence of the visible light image to showcase the continuous, uninterrupted movement of the target object along the continuous transmission path in a way that correlates the structural and shape features of the target object. Thus, the correlation presentation of the structural and shape features of the target object in the inspection video helps inspectors improve the efficiency of object matching in identifying target objects among physical objects.

[0176] In order to enhance the correlation between the structural features and shape features of the same target object by using position matching of the structural features and shape features, while taking into account the intuitive visibility of the structural features and shape features respectively, the image fusion module 1430 can fuse the part structural features of the target object that has been transmissively scanned by the transmission scanning component and is still in the channel segment in the visible light image at the imaging time within the fusion time window. The fusion time window is used to characterize the time range from when the target object arrives at the scanning position of the transmission scanning component to when the target object moves out of the channel segment.

[0177] For example, the video generation apparatus in this embodiment may further include a time prediction module (not included in...). Figure 14 As shown in the diagram, this method is used to: when a target object located on the outer side of the transmission scanning channel entrance is detected, determine a fusion time window based on the transmission speed of the transmission mechanism and the distance between the target object's current position and the scanning position. For the detection method of the target object located on the outer side of the transmission scanning channel entrance, please refer to the aforementioned embodiments, which will not be repeated here.

[0178] Furthermore, the process of associating the structural features and shape features of the target object in the over-inspection video can be configured as described in the foregoing embodiments. Figure 4 As shown, during the period when any part of the target object becomes a scanned part of interest, the presentation state of that part in the over-inspection video changes from being highlighted by the corresponding part's shape features to being highlighted by the part's structural features. Furthermore, the presentation state of that part switches back to being highlighted by the corresponding part's shape features in response to a change in the position of the segment that moves out of the channel.

[0179] To ensure positional matching between the structural features and shape features of the scanned region of interest of the same target object, the structural features fused into the visible light image can be dynamically refreshed in response to the continuous movement of the target object. Furthermore, the image fusion module 1430 can fuse the dynamically refreshed structural features into the visible light image based on the time matching between the refresh time of the dynamic refresh of the structural features and the imaging time of the visible light image. This ensures that the fused structural features in the visible light image are refreshed at a refresh time that matches the imaging time of the visible light image. The method for determining the imaging time of the visible light image can be found in the aforementioned embodiments and will not be repeated here.

[0180] For example, the over-detection video can be generated by encoding the image sequence of the visible light image frame by frame, and the image fusion module 1430 can determine a matching refresh time that matches the imaging time of the visible light image of the current frame to be encoded in response to the arrival of the encoding time of each frame of the visible light image; and construct features of the parts that will be refreshed at the determined matching refresh time and fuse them into the visible light image of the current frame to be encoded.

[0181] When the transmission scanning component is configured to perform line scanning on the target object, the image fusion module 1430 can determine the matching refresh time using the methods described in the aforementioned embodiments. Figure 6 The method is as follows: In the refresh time sequence that matches the scanning frequency, the refresh time that is closest to the imaging time of the visible light image of the current frame to be encoded is determined as the matching refresh time that matches the imaging time of the visible light image of the current frame to be encoded. Since the particulate structural features are dynamically refreshed at the scanning frequency of the transmission scanning component (i.e., the scanning frequency of the line scan), the frame rate of the visible light image sequence is different from the scanning frequency. Therefore, the determined matching refresh time may be exactly the same as the imaging time of the visible light image of the current frame to be encoded. Furthermore, if the determined matching refresh time is later than the encoding time of the visible light image of the current frame to be encoded, the fusion and encoding of the visible light image of the current frame to be encoded can be delayed until the determined matching refresh time arrives.

[0182] To further enhance the intuitive visibility of structural features and shape features, and to minimize the time consumed in the fusion processing of visible light images, in the embodiments of this application, the image fusion module 1430 can fuse a gradient mask layer generated based on the part structural features frame by frame in the visible light image located within the fusion time window at the imaging time. The part structural features are dynamically refreshed in response to the continuous movement of the target object, and the gradient mask layer is triggered to be generated in response to each fusion, so as to ensure that each superimposed gradient mask layer is generated by the part structural features that are refreshed at the matching refresh time. Thus, the highlighting of part structural features is configured such that the part structural features represent the coverage and occlusion of the part shape features of the same part.

[0183] For example, if the transmission scanning component is configured to perform line scanning on a target object, the transmission scanning data generated by the transmission scanning component can include line scan data generated for each line scan. This line scan data can be used to generate a transmission ray strip image. The generated transmission ray strip image can be managed using a strip buffer queue, which can be deployed in a data buffer module further included in the video generation apparatus (not shown in the original text). Figure 14 As shown in the figure, the method of managing transmitted X-ray strip images using a strip buffer queue can be found in the foregoing embodiments. Figure 7The explanation will not be repeated here; furthermore, the gradient mask layer may include transmission ray strip images arranged and stitched in the order of scanning time, wherein: the transmission ray strip images generated based on the line scan data of each line scan are processed with pseudo-color to colorize the pixels of the feature content used to characterize the structural features of the part in the transmission ray strip images, and the pixels in the transmission ray strip images that match the full-load background correction data of the road segment in the channel are set to empty pixels with transparent rendering effect.

[0184] In this case, in order to perform a fusion operation of overlaying the gradient mask layer onto the visible light image when it contains the structural features, and at the same time to avoid performing a fusion operation of overlaying the gradient mask layer onto the visible light image when it is globally transparent, the image fusion module 1430 can be further configured to: initiate the fusion of the structural features in the visible light image when it is detected that the data content contained in the transmission scan data changes from the full-load background correction data of the channel segment to the feature content of the structural features; and stop the fusion of the structural features in the visible light image when it is detected that the channel segment becomes empty due to the removal of the target object.

[0185] Furthermore, the image fusion module 1430 can superimpose a gradient mask layer on a visible light image in the following manner: superimpose the gradient mask layer on the calibration image region of each frame of visible light image located within the fusion time window at the imaging time. The calibration image region is the mapping region of the road segment interval between the scanning position and the exit side boundary of the road segment in the channel in the visible light image. Moreover, the layer size of the gradient mask layer matches the region size of the calibration image region.

[0186] Figure 15 For example Figure 14 The video generation apparatus in the illustrated embodiment further supports an extended structure diagram of a manual switching presentation mechanism. Please refer to... Figure 15 If the video generation device in this embodiment is further configured to support a manual switching presentation mechanism, then the video generation device may further include a signal monitoring module 1450 for monitoring the signal status of the switching control signal, and the image fusion module 1430 may further be used for:

[0187] In response to the signal state change of the switching control signal, the overall structural features of the target object are fused in the visible light image where the imaging time is later than the fusion time window. The fusion position of the overall structural features of the target object in the visible light image is matched with the image position of the overall shape features of the target object in the visible light image, so that the presentation state of the target object in the outside image area of ​​the outside segment of the transmission scanning channel corresponding to the exit of the over-inspection video is switched from the overall shape features to the overall structural features.

[0188] In response to the signal state reset of the switching control signal, the fusion of the overall structural features is stopped, so that the presentation state of the target object in the exit area of ​​the over-inspection video switches back to the overall shape features.

[0189] The specific implementation of the above-mentioned functions further added by the image fusion module 1430 to support manual switching of the presentation mechanism can be referred to the foregoing embodiments regarding... Figure 8 The description will not be repeated here.

[0190] Figure 16 For example Figure 14 The video generation apparatus in the illustrated embodiment further supports an extended structure diagram of an intelligent recognition mechanism. Please refer to [link to diagram]. Figure 16 If the video generation device in this embodiment is further configured to support an intelligent recognition mechanism, then the video generation device may further include an intelligent recognition module 1470, used to perform intelligent feature recognition on the structural features of the target object (e.g., structural features of the scanned area of ​​interest); and, in response to the abnormal recognition result generated by the intelligent feature recognition, to add a visual abnormality alarm tag frame by frame in the image sequence of the visible light image, wherein the position of the visual abnormality alarm tag matches the structural feature or shape feature in the visible light image, so that the visual abnormality alarm tag moves synchronously in response to the continuous movement of the target object in the over-inspected video.

[0191] The structural features (e.g., structural features of the scanned area of ​​interest) identified by the intelligent recognition module 1470 can be provided by the image fusion module 1430, or obtained by the intelligent recognition module 1470 using transmission scan data. Furthermore, the specific functional implementation of the intelligent recognition module 1470 can be referred to the foregoing embodiments regarding... Figure 9 The description will not be repeated here.

[0192] In addition, such as Figure 15 He Ru Figure 16 The two extended structures can be combined, and the combined structure will not be described here.

[0193] In another embodiment of this application, a non-transitory computer-readable storage medium is also provided, which stores instructions that, when executed by a processor, cause the processor to perform the video generation method described in the foregoing embodiments.

[0194] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method of video generation, the method comprising: The method comprises: generating a visible light image using image acquisition data generated by a visible light imaging assembly; wherein the visible light imaging assembly is configured to image an in-channel path segment of the conveying mechanism located inside a channel of a transmission scanning channel, and an out-of-channel path segment located outside an entrance and exit of the transmission scanning channel, the visible light image comprising a continuous conveying path segment of the in-channel path segment and the out-of-channel path segment successively connected to each other, and an over-check video generated based on an image sequence of the visible light image is used to present a moving process of a target object along the continuous conveying path segment; in the visible light image located in a fusion time window at an imaging time, frame-by-frame fusion of a site configuration feature of the target object that has been arranged in the transmission scanning assembly transmission scanning of the transmission scanning channel inside the channel and is still in the scanned attention site of the in-channel path segment, so as to fuse the configuration feature of the target object in the image sequence of the visible light image using transmission scanning data generated by the transmission scanning assembly on the target object passing through the in-channel path segment; wherein the fusion time window is used to represent a time range from the target object reaching a scanning position of the transmission scanning assembly to the target object moving out of the in-channel path segment, the configuration feature is matched with the appearance feature position of the target object included in the visible light image, so that the continuous moving process of the target object presented by the over-check video includes an associated presentation process in which the configuration feature and the appearance feature are synchronously and gradually changed in response to the continuous movement of the target object; wherein the associated presentation process is configured such that, during the period when any site of the target object becomes the scanned attention site, the presentation state of the site in the over-check video is switched from being highlighted by the corresponding site appearance feature to being highlighted by the site configuration feature, and the presentation state of the site is switched back to being highlighted by the corresponding site appearance feature in response to the change in position moving out of the in-channel path segment; wherein the process of frame-by-frame fusion of the site configuration feature in the visible light image comprises: frame-by-frame fusion of a gradual change mask layer generated based on the site configuration feature in the visible light image located in the fusion time window at the imaging time; the site configuration feature is dynamically refreshed in response to the continuous movement of the target object, and the gradual change mask layer is triggered to be generated in response to each fusion, so that the site configuration feature highlighting is configured such that the site configuration feature represents the covering and shielding of the site appearance feature of the same site.

2. The video generation method according to claim 1, wherein the visible light imaging assembly comprises a first camera assembly, a second camera assembly and a third camera assembly, wherein the imaging field of view of the first camera assembly covers the in-channel path segment, and the imaging field of view of the second camera assembly and the third camera assembly respectively covers the out-of-channel path segment outside the entrance and exit of the transmission scanning channel. The image acquisition data generated by the visible light imaging assembly includes an in-channel image sequence generated by the first camera assembly, and out-of-channel image sequences generated by the second camera assembly and the third camera assembly; Generating a visible light image based on the image acquisition data generated by the visible light imaging assembly includes: splicing based on synchronous frames in the in-channel image sequence and the out-of-channel image sequence to generate the visible light image.

3. The video generation method of claim 1, further comprising: determining the fusion time window based on a conveying speed of the conveying mechanism and a distance between a current position of the target object and the scanning position when detecting the target object located outside an entrance of the in-channel path of the transmission scanning channel; and / or, starting fusion of the part construction feature when detecting that data content contained in the transmission scanning data changes from full-load background correction data of the in-channel path to feature content of the part construction feature; and / or, stopping fusion of the part construction feature when detecting that the in-channel path becomes an empty state due to the target object moving out.

4. The video generation method of claim 1, wherein: the part construction feature is dynamically refreshed in response to continuous movement of the target object; fusing, in the visible light image located in the fusion time window at an imaging time, the part construction feature of the scanned attention part of the target object that has been scanned by the transmission scanning assembly and is still in the in-channel path, frame by frame, includes: fusing the part construction feature dynamically refreshed at a refresh time matching the imaging time of the visible light image into the visible light image, so that the part construction feature fused in the visible light image is refreshed at a refresh time matching the imaging time of the visible light image.

5. The video generation method of claim 4, wherein: the over-check video is generated by frame-by-frame encoding of an image sequence of the visible light image; fusing the part construction feature dynamically refreshed at a refresh time matching the imaging time of the visible light image into the visible light image includes: determining a matching refresh time matching the imaging time of the visible light image of a current frame to be encoded in response to an encoding time of each frame of the visible light image arriving; fusing the part construction feature refreshed at the matching refresh time into the visible light image of the current frame to be encoded.

6. The video generation method of claim 5, wherein: the part construction feature is dynamically refreshed at a scanning frequency of the transmission scanning assembly, and a frame rate of the visible light image sequence is different from the scanning frequency; determining a matching refresh time matching the imaging time of the visible light image of a current frame to be encoded includes: In a refresh time sequence matching the scanning frequency, a refresh time closest to the imaging time of the visible light image of the current frame to be encoded is determined as the matching refresh time, wherein if the matching refresh time is later than the encoding time of the visible light image of the current frame to be encoded, the fusion and encoding of the visible light image of the current frame to be encoded are delayed until the matching refresh time arrives.

7. The video generation method of claim 1, wherein, a field of view direction of the visible light imaging component is the same as a transmission ray direction of the transmission scanning component.

8. The video generation method of claim 1, wherein, in the visible light image whose imaging time is within the fusion time window, the gradual mask layer generated based on the part structure feature is fused frame by frame, including: superimposing the gradual mask layer on a calibrated image region of each frame of the visible light image whose imaging time is within the fusion time window, the calibrated image region being a mapping region of a road segment interval between the scanning position and an outlet side boundary of the road segment in the channel in the visible light image, and a layer size specification of the gradual mask layer matching a region size specification of the calibrated image region.

9. The video generation method of claim 1, wherein, the transmission scanning of the target object by the transmission scanning component is configured as line scanning; the transmission scanning data includes line scanning data generated by each line scanning, the line scanning data being used to generate a transmission ray strip image; the gradual mask layer includes the transmission ray strip images spliced in a scanning time sequence, the transmission ray strip image generated based on the line scanning data of each line scanning being pseudo-color processed, so that pixels in the transmission ray strip image representing feature content of the part structure feature are colorized, and pixels in the transmission ray strip image matching full load background correction data of the road segment in the channel are set as empty pixels with a transparent presentation effect.

10. The video generation method of claim 9, wherein, the refresh of the part structure feature is a queue refresh of a strip cache queue used to store the transmission ray strip images, wherein: the queue refresh is triggered in response to the transmission ray strip image storage caused by each line scanning; the strip cache queue is configured as a first-in-first-out queue, so that the transmission ray strip images in the strip cache queue are arranged in a scanning time sequence, and a queue position of the transmission ray strip image in the strip cache queue is shifted in a queue-out direction matching a moving direction of the target object in response to each queue refresh; a queue length of the strip cache queue is configured so that a sum of widths of a predetermined number of the transmission ray strip images matches a first layer size of the gradual mask layer in a layer width direction corresponding to the moving direction of the target object. The length of the transmitted ray strip image matches a second layer size of the gradient mask layer in the layer height direction; In the generated gradient mask layer triggered in response to each fusion, the transmitted ray strip image stored by the strip cache queue at the time of the fusion is included.

11. The video generation method of claim 1, wherein The method further comprises: In response to a signal state change of the switching control signal, the overall configuration feature of the target object is fused in the visible light image at an imaging time later than the fusion time window, wherein a fusion position of the overall configuration feature in the visible light image matches an image position of the overall contour feature of the target object in the visible light image, so that a presentation state of the target object in the exit-out picture area of the channel-out path segment outside the exit of the transmission scanning channel in the over-check video switches from the overall contour feature to the overall configuration feature; In response to a signal state reset of the switching control signal, the fusion of the overall configuration feature is stopped, so that the presentation state of the target object in the exit-out picture area switches back to the overall contour feature.

12. The video generation method of claim 1, wherein, The method further comprises: performing feature intelligent identification on the configuration feature; In response to an abnormal identification result generated by the feature intelligent identification, a visual abnormal alarm label is added frame by frame in the image sequence of the visible light image, wherein the visual abnormal alarm label matches the configuration feature or the contour feature in the visible light image, so that the visual abnormal alarm label moves synchronously in the over-check video in response to continuous movement of the target object.

13. A video generating apparatus, comprising: The method further comprises: a video generation module configured to generate a visible light image based on image acquisition data generated by a visible light imaging assembly, wherein the visible light imaging assembly is configured to image a channel-in path segment inside a transmission scanning channel of a transmission mechanism and a channel-out path segment outside the transmission scanning channel, the visible light image includes a continuous transmission path segment formed by the channel-in path segment and the channel-out path segment, and an over-check video generated based on an image sequence of the visible light image is used to present a movement process of a target object along the continuous transmission path segment. The image fusion module is configured to fuse, in the visible light image located within the fusion time window at the imaging moment, the part structure feature of the scanned attention part of the target object, which has been arranged inside the channel of the transmission scanning channel and is still in the scanning position, frame by frame, to fuse the structure feature of the target object in the image sequence of the visible light image by using the transmission scanning data of the target object passing through the in-channel road section generated by the transmission scanning assembly. The fusion time window is configured to represent the time range from the target object reaching the scanning position of the transmission scanning assembly to the target object moving out of the in-channel road section, and the structure feature is matched with the position of the contour feature of the target object included in the visible light image, so that the continuous movement process of the target object presented in the over-check video includes an associated presentation process in which the structure feature and the contour feature are synchronously and gradually changed in response to the continuous movement of the target object. The associated presentation process is configured to: during the period in which any part of the target object becomes the scanned attention part, the presentation state of the part in the over-check video is switched from being highlighted by the corresponding part contour feature to being highlighted by the part structure feature, and the presentation state of the part is switched back to being highlighted by the corresponding part contour feature in response to the position change of moving out of the in-channel road section. The process of fusing the part structure feature frame by frame in the visible light image includes: fusing a gradual mask layer generated based on the part structure feature frame by frame in the visible light image located within the fusion time window at the imaging moment; the part structure feature is dynamically refreshed in response to the continuous movement of the target object, and the gradual mask layer is triggered to be generated in response to each fusion, so that the part structure feature highlighting is configured to cover and shield the part contour feature of the same part.

14. The video generation device of claim 13, further comprising an intelligent recognition module configured to perform feature intelligent recognition on the structure feature, and in response to an abnormal recognition result generated by the feature intelligent recognition, add a visual abnormal alarm label frame by frame in the image sequence of the visible light image, wherein the visual abnormal alarm label is matched with the position of the structure feature or the contour feature in the visible light image, so that the visual abnormal alarm label is synchronously moved in the over-check video in response to the continuous movement of the target object.

15. A detection system characterized by, The video generation device comprises: a transmission scanning channel; a conveying mechanism configured to convey the target object through the transmission scanning channel; a visible light imaging assembly configured to image the in-channel road section of the conveying mechanism located inside the channel of the transmission scanning channel, and the out-of-channel road section located outside the entrance and exit of the transmission scanning channel; a transmission scanning assembly configured to transmit scan the target object passing through the in-channel road section. a processing module configured to perform the video generation method of any one of claims 1-12; a communication component configured to establish a communication connection between the processing module and a video playing device, so that the overchecked video is transmitted to the video playing device for playing and presenting.

16. A video playback device, comprising: comprising: a display module; a processing module configured to perform the video generation method of any one of claims 1-12, and play and present the overchecked video through the display module; a communication component configured to establish a communication connection between the processing module and a detection system comprising the transmission scanning channel, the conveying mechanism, the visible light imaging component, and the transmission scanning component, the communication connection being used for the processing module to acquire the image acquisition data and the transmission scanning data.

17. A non-transitory computer-readable storage medium, comprising: The non-transitory computer-readable storage medium stores instructions that, when executed by a processor, cause the processor to perform the video generation method of any one of claims 1-12.

Images