A distributed scene-based intelligent shooting method

Abstract

The application provides a kind of distributed scene intelligent shooting method, it is related to intelligent shooting technical field.The method includes: the camera of perception and collection node obtains the re-identification feature vector of user;Through the user identity node broadcast bluetooth beacon frame, the bluetooth scanning module of perception and collection node receives bluetooth beacon frame and extracts user identification;Re-identification feature vector is associated with user identification binding;When identifying the user after binding arrives predetermined scene node, scene node sends the trigger signal carrying scene type identification;According to type identification, the scene image of user interaction is generated by shooting.The application associates and binds re-identification feature vector with the user identification carried by bluetooth beacon frame, so that the same user has a unified identity anchor in each scene node, solves the problem that the same user needs to repeatedly scan code verification in different scene nodes, image data cannot be cross-node associated aggregation, and the system cannot automatically match and generate strategy according to scene type.

Description

Technical Field

[0001] This invention relates to the field of intelligent shooting technology, and in particular to a distributed scene-based intelligent shooting method. Background Technology

[0002] Currently, distributed intelligent photography systems have emerged in cultural and tourism scenic spots, commercial streets, and offline brand spaces, which provide users with automated shooting and interactive services by deploying cameras and interactive screens in multiple locations.

[0003] When users first use this type of system, they typically need to register by scanning a QR code or at a card issuing machine to enter their identity into the system. Subsequently, the image assets acquired at each shooting location are bound to this identity. Each shooting location independently records its own collected image data, and data ownership is based on the local records generated by that shooting location itself.

[0004] However, when a user moves from the first shooting point to the second shooting point, the second shooting point cannot obtain the user's registered identity information from the first shooting point. The user needs to scan the code again or re-verify their identity before the second shooting point can associate the newly generated image data with the user's identity. Because there is no shared user identity anchoring mechanism between shooting points, image data generated by the same user at different shooting points is recorded separately at each node. The system cannot perform cross-node association and aggregation of image data scattered across different nodes according to a unified user dimension, resulting in the user having to repeat the identity verification operation every time they arrive at a new shooting point. Simultaneously, the existing system lacks the ability to perceive the physical environment during image generation and cannot adjust the generation strategy in real time based on information such as the type of scene node the user is currently in or the advertising content being played. Summary of the Invention

[0005] The embodiments of the present invention provide a distributed scene-based intelligent shooting method, which aims to solve the problems that the same user needs to repeatedly scan codes for verification at different scene nodes, that image data cannot be associated and aggregated across nodes, and that the system cannot automatically match and generate strategies according to scene type.

[0006] To achieve the above objectives, the present invention provides a distributed scene-based intelligent shooting method, comprising the following steps: The user's re-identification feature vector is obtained through the cameras at the sensing and acquisition nodes; Users broadcast Bluetooth beacon frames through a user identity node they carry with them, and receive the Bluetooth beacon frames through the Bluetooth scanning module of the sensing and acquisition node, extracting the user identifier stored in the user identity node; The re-identification feature vector is associated and bound with the user identifier; When the camera recognizes that a bound user has arrived at a predetermined scene node, the scene node sends a shooting trigger signal, which carries the type identifier of the scene node; Based on the type identifier, capture and generate contextualized images for user interaction.

[0007] Further, associating and binding the re-identification feature vector with the user identifier includes: The re-identification feature vector is compared with the registration feature vector in the authorized user database; If the comparison matches, the user corresponding to the re-identified feature vector is marked as an authorized user, and the user identifier is bound to the authorized user; If the comparison is inconsistent, the user corresponding to the re-identification feature vector will be marked as an unregistered user, and the corresponding image data will be marked as a temporary cache state, and deleted or blurred after the preset life cycle is reached.

[0008] Furthermore, after the user identity node broadcasts the Bluetooth beacon frame, it also includes: The sensing and acquisition node receives the Bluetooth beacon frame via its Bluetooth angle-of-arrival antenna array, calculates the angle of arrival of the user relative to the sensing and acquisition node based on the phase difference of the Bluetooth beacon frame arriving at each antenna element, calculates the distance value based on the signal strength, and determines the user's spatial coordinates using the angle of arrival and the distance value.

[0009] Furthermore, the scene nodes include one or more of the following: The trigger signal for the street photography node is generated by the camera of the sensing and acquisition node when it recognizes that the bound user has entered the predetermined framing range; The trigger signal for the blind box card drawing node is generated by the sensor configured in the scene node when a trigger event is detected; The corresponding trigger signal for the advertising interaction node is generated by the smart screen node when the advertising material is played to the predetermined key frame; The trigger signal for the designated photography node is generated when the scene node establishes a connection with the user identity node through near-field communication. The trigger signal for the fixed check-in node is generated when the camera of the sensing and acquisition node recognizes the user's predetermined action.

[0010] Furthermore, before the advertising interaction node sends the trigger signal, it also includes: The sensing and acquisition node performs visual verification on the current cached image. The visual verification includes detecting whether the user is facing the smart screen node and detecting whether the current cached image contains the display content of the advertising material. The trigger signal is allowed to be sent only if both tests pass.

[0011] Furthermore, when the scene node is an advertising interaction node, the step of capturing and generating a scene-based image for user interaction based on the type identifier includes: The sensing and acquisition node extracts the image at the corresponding moment from the image cache according to the timestamp carried by the trigger signal, and sends the image to the cloud server; The cloud server obtains the metadata of the advertising material, which includes brand identity and product category; The cloud server performs portrait segmentation on the image to determine the editable area, fills the brand logo and product category into the prompt template, and performs image redrawing within the editable area according to the prompt template to generate a virtual object containing the brand logo, thereby obtaining the output image.

[0012] Furthermore, when the scene node is a blind box card-drawing node, the step of capturing and generating a scene-based image for user interaction based on the type identifier includes: The image at the corresponding moment is extracted from the image buffer according to the trigger signal; A random seed is generated using the session identifier, device identifier, and current timestamp; The sampling probability of each style entry is calculated based on the weight and cooldown number of each style entry in the style card pool, and the target style is determined by sampling in the sampling probability distribution using the random seed. The image is stylized and redrawn according to the target style to obtain the output image.

[0013] Furthermore, when the scene node is a street photography node, the trigger signal is generated by the camera of the sensing and acquisition node in the following manner: Define a virtual fence area in the view coordinate system of the camera; The user's spatial coordinates are determined via a Bluetooth angle-of-arrival antenna array; The trigger signal is generated when the user's spatial coordinates fall within the virtual fence area.

[0014] Furthermore, when the scene node is a designated photography node, the step of capturing and generating a scene-based image for user interaction based on the type identifier includes: The scene node acquires the voice commands sent by the user through the terminal device; The voice commands are parsed into gimbal control parameters and image processing parameters; The camera orientation of the sensing and acquisition node is adjusted according to the gimbal control parameters, and the acquired images are optimized according to the image processing parameters to obtain the output image. Establish the association between the output image and the user identifier.

[0015] Furthermore, after capturing and generating contextualized images for user interaction, the process also includes: Using the user identifier as an index, all captured and generated images associated with the target user are retrieved; The recalled images are sorted according to the shooting time to generate a spatiotemporal image sequence of the target user; The spatiotemporal image sequence is pushed to the target user's terminal device.

[0016] The above technical solution has the following technical effects: First, by associating re-identification feature vectors with user identifiers, the same user can have a unified identity anchor across different scene nodes. After a user enters any scene node, the camera extracts the re-identification feature vector of the current pedestrian in real time and compares it with the authorized user database. Once matched, it is automatically associated with the bound user identifier, eliminating the need for the user to actively present identity credentials or repeatedly perform verification operations, thus fundamentally eliminating the barrier to identity attribution across nodes.

[0017] Second, scene nodes send trigger signals carrying type identifiers, and the cloud executes the corresponding image generation strategy based on the type identifier. The advertising interaction node is triggered when the advertising material reaches a predetermined keyframe, generating a virtual object containing the brand identifier in the image; the blind box card-drawing node is triggered by sensor detection of the event, using random seeds to sample from the style card pool to determine the target style and perform stylized redrawing; the street photography node is triggered when a user is detected entering a predetermined framing range, directly outputting the original image. Each scene node is independent in its triggering method and generation strategy, uniquely determined by the type identifier.

[0018] Third, the camera continuously captures image sequences at a fixed sampling rate and writes them to the buffer, with the trigger signal carrying a timestamp. After receiving the trigger signal, the sensing and acquisition nodes extract the image corresponding to the time from the buffer based on the timestamp, rather than starting the shutter only after receiving the trigger signal. This eliminates the link delay from the occurrence of the interaction to the transmission of the trigger signal and then to the camera's response, ensuring that the captured image and the user's actual interaction correspond precisely in the time dimension.

[0019] Fourth, establish a relationship between output images and user identifiers. Use the user identifier as an index to retrieve all captured and generated images associated with the target user, sort them by capture time, and push them out uniformly. Images generated by a user at any scene node are associated with the same user identifier. A single query can obtain a complete image sequence across all nodes without needing to access or download each node separately.

[0020] Fifth, images that do not match the re-identified feature vectors in the authorized user database are temporarily stored in a temporary cache with a set lifespan, and are automatically deleted or blurred upon arrival. Images without matching user identifiers do not enter the subsequent upload and generation process, ensuring that unauthorized data is cleaned up before ownership is confirmed, and preventing images of unregistered users from entering the cloud or being obtained by other accounts. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating a distributed scene-based intelligent shooting method according to an embodiment of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0023] Figure 1 This is a flowchart illustrating a distributed scene-based intelligent shooting method according to an embodiment of the present invention. Figure 1 As shown, the method provided in this application embodiment includes the following steps: Step S101: Obtain the user's re-identification feature vector through the camera of the sensing and acquisition node.

[0024] In this embodiment, the sensing and acquisition node is an image acquisition device deployed in the target space, such as a cultural and tourism scenic area, a commercial street, or a brand offline experience space. The sensing and acquisition node includes a camera, a processor, and a communication module. The camera is used to acquire images of a predetermined area, the processor is used to process the acquired images, and the communication module is used to interact with other nodes or a cloud server.

[0025] Specifically, the cameras at the sensing and acquisition nodes continuously capture image sequences within their field of view at a predetermined sampling frequency, such as 30 frames per second. The processor extracts pedestrian re-identification (ReID) feature vectors from the acquired images.

[0026] Re-identification feature vectors are digital codes used to characterize the appearance of pedestrians. Re-identification technology is a cross-camera pedestrian retrieval technique in computer vision, with the core objective of identifying the same pedestrian in images captured by different cameras. In its implementation, the processor inputs the acquired pedestrian images into a pre-trained re-identification neural network model, such as a pedestrian re-identification model based on convolutional neural networks or Transformer architectures. This model extracts deep features from the images and outputs a high-dimensional feature vector. This feature vector is an abstract mathematical expression of the pedestrian's appearance attributes, including but not limited to visual features such as clothing color, body shape, and carried items. The training objective of the re-identification neural network model is that the feature vectors extracted from the same pedestrian under different cameras and poses should be close in distance in the vector space, while the feature vectors of different pedestrians should be far apart. Through this technique, the system can identify and track specific users based solely on their appearance features without relying on active user cooperation, such as facing the camera or presenting identification.

[0027] Step S102: The user broadcasts a Bluetooth beacon frame through the user identity node carried by the user, receives the Bluetooth beacon frame through the Bluetooth scanning module of the sensing and acquisition node, and extracts the user identifier stored in the user identity node.

[0028] In this embodiment, the user identity node is a physical device carried by the user, which may take the form of a smart card, bracelet, or patch. The user identity node internally stores an encrypted user identifier (User_UUID), which is a unique identifier for the user within the system.

[0029] The user identity node integrates a Bluetooth Low Energy (BLE) Beacon module. This module broadcasts Bluetooth beacon frames at a predetermined frequency, such as once every 100 milliseconds. These beacon frames carry an encrypted user identifier. BLE Beacon technology is a broadcast positioning and sensing technology based on Bluetooth Low Energy, characterized by low power consumption and wide coverage, making it suitable for presence sensing and proximity detection in indoor scenarios.

[0030] The sensing and acquisition node integrates a BLE scanning module. This module continuously monitors surrounding Bluetooth broadcast signals. When a user carrying a user identity node enters the BLE scanning range of the sensing and acquisition node, the BLE scanning module receives the Bluetooth beacon frame broadcast by the user identity node and extracts the encrypted user identifier from it. The extracted user identifier is then passed to the processor of the sensing and acquisition node for further processing.

[0031] Through the aforementioned BLE broadcasting and scanning mechanism, the system can detect the user's presence and obtain their identity identifier in advance, even before the user actively interacts with any interactive device, thus providing a data foundation for subsequent identity association and contextualized services.

[0032] Step S103: Associate and bind the re-identification feature vector with the user identifier.

[0033] In this embodiment, the processor of the sensing and acquisition node establishes a correspondence between the re-identification feature vector extracted in step S101 and the user identifier extracted in step S102, forming a cross-modal association between visual identity and radio frequency identity.

[0034] Specifically, the processor performs spatiotemporal correlation matching between the re-identification feature vectors of all pedestrians extracted from camera images within a predetermined time window and all user identifiers received by the BLE scanning module within the same time window. The correlation matching is based on the proximity between the spatial position of the pedestrian corresponding to the re-identification feature vector within the camera's field of view and the user's spatial position indicated by the BLE beacon frame signal strength. When the pedestrian position corresponding to a certain re-identification feature vector and the beacon position corresponding to a certain user identifier are spatially close enough (e.g., less than 1 meter apart) and temporally synchronized, the processor binds the re-identification feature vector to that user identifier.

[0035] Once the binding relationship is established, the user has a unified identity anchor across the entire system. Thereafter, regardless of where the user moves, the camera at that node can automatically identify the user by extracting the current pedestrian's re-identification feature vector and comparing it with the bound feature vector, without requiring the user to repeatedly present identification credentials or perform any manual verification. This cross-modal association mechanism is a core technical means to solve the problems of data fragmentation across different nodes and the need for users to repeatedly scan codes.

[0036] In one optional implementation of this embodiment, step S103 specifically includes the following steps: The re-identification feature vector is compared with the registration feature vector in the authorized user database. The authorized user database is a database maintained by a cloud server, which stores the registration feature vectors of registered users and their corresponding user identifiers. When a user uses the system for the first time, they usually need to register by scanning a QR code or at a card issuing machine. The system collects one or more reference images of the user and extracts their re-identification feature vectors as registration feature vectors, storing them in the authorized user database. At the same time, the user identifier in the user identity node is bound to this registration feature vector.

[0037] If the comparison is successful, that is, if the currently extracted re-identification feature vector is successfully matched with a certain registered feature vector in the authorized user database, such as if the vector cosine similarity is greater than a predetermined threshold, such as 0.85, then the pedestrian corresponding to the re-identification feature vector is marked as an authorized user, and the user identifier extracted in step S102 is bound to the authorized user.

[0038] If the comparison fails—that is, if the currently extracted re-identification feature vector does not match any of the registered feature vectors in the authorized user database—the pedestrian corresponding to the re-identification feature vector is marked as an unregistered user, and the corresponding image data is marked as temporarily cached. This cached data will be deleted or blurred after a preset lifespan, such as 3 days. This mechanism ensures that the image data of unregistered users is not permanently stored or included in subsequent generation processes, thus protecting the privacy of unregistered users at the data level.

[0039] Step S104: When the camera recognizes that the bound user has arrived at the predetermined scene node, the scene node sends a shooting trigger signal, which carries the type identifier of the scene node.

[0040] In this embodiment, multiple scene nodes are deployed in the target space. Scene nodes are physical devices or combinations of devices used for specific scenario-based interactions with users. Different types of scene nodes provide different types of interactive services and image generation services. The types of scene nodes include, but are not limited to: street photography nodes, blind box drawing nodes, advertising interaction nodes, dedicated photography nodes, and fixed-point check-in nodes.

[0041] The cameras at the sensing and acquisition nodes track the location and posture of users with bound identities in real time. When a camera detects that a user has entered a predetermined interaction area of ​​a scene node, the scene node generates and sends a trigger signal to capture the image. This trigger signal carries a type identifier for the scene node, which uniquely indicates the category of the scene node. For example, 01 represents a street photography node, 02 represents a blind box card-drawing node, and 03 represents an advertising interaction node, etc.

[0042] Step S105: Capture and generate contextualized images for user interaction based on the type identifier.

[0043] In this embodiment, after receiving the trigger signal, the sensing and acquisition node extracts the user image at the corresponding time from the image cache according to the timestamp carried in the trigger signal, associates the image with the scene type identifier, and sends it to the cloud server.

[0044] After receiving the image and type identifier, the cloud server queries the policy library using the type identifier as an index. The policy library is a configuration database maintained by the cloud server, which pre-stores generation policy configurations that correspond one-to-one with each scene type identifier. Each generation policy configuration includes at least: a prompt template, model parameters, redraw intensity, editable region constraint rules, and output specifications.

[0045] Based on the retrieved generation strategy configuration, the cloud server invokes the corresponding AIGC model to perform image generation or image redrawing on the image, obtaining an output image that matches the current scene type. The generation strategy configurations for different scene types are independent of each other, so the image content and style obtained by the same user at different scene nodes are different, realizing scene-differentiated intelligent photography services.

[0046] Finally, the cloud server establishes a relationship between the output image and the user identifier, and records the output image under the user's name for subsequent cross-node asset aggregation and unified delivery.

[0047] Example 2 This embodiment provides a method for determining user spatial coordinates in a distributed scene-based intelligent shooting method.

[0048] In this embodiment, the sensing and acquisition node also integrates a Bluetooth Angle of Arrival (AoA) antenna array. The Bluetooth AoA antenna array consists of multiple antenna elements arranged in a predetermined geometry, such as a linear array or a planar array. When the user identity node broadcasts a Bluetooth beacon frame, the radio waves of the beacon frame propagate in the form of spherical waves, and there is a slight difference in the arrival time of different antenna elements. This time difference manifests as a phase difference in the beacon frame received by each antenna element.

[0049] The processor at the sensing and acquisition node calculates the angle of incidence of the radio wave based on the phase difference of the Bluetooth beacon frame arriving at each antenna element. This angle is the angle of arrival of the user relative to the sensing and acquisition node. Simultaneously, the processor calculates the distance between the user and the sensing and acquisition node based on the difference between the Received Signal Strength Indication (RSSI) and the calibrated transmit power value, combined with a path loss model of radio waves in free space.

[0050] By using the angle of arrival and distance as polar coordinate parameters, the user's spatial coordinates in a polar coordinate system with the sensing and acquisition nodes as the origin can be determined. If spatial coordinates in a rectangular coordinate system are required, they can be calculated using the polar-to-rectangular coordinate conversion formula. Through this technique, the system can achieve sub-meter-level user positioning accuracy in indoor environments without GPS signals, providing accurate spatial basis for determining whether a user has entered the interactive area of ​​a specific scene node.

[0051] Example 3 This embodiment provides specific implementation methods for five scene nodes.

[0052] Scene Node 1: Street Photography Node Street photography nodes are scene nodes used to capture images without being noticed while a user is walking naturally.

[0053] In this embodiment, the trigger signal for the street photography node is generated by the camera of the sensing and acquisition node when it detects a bound user entering a predetermined framing range. Specifically, the processor of the sensing and acquisition node predefines a virtual fence area in the camera's view coordinate system. The virtual fence can be a polygonal area such as a rectangle, hexagon, or irregular polygon drawn in the view coordinate system by an algorithm, or it can be a visual boundary defined by specific physical markers such as ground stickers, changes in the color of paving tiles, or physical door frames identified by computer vision.

[0054] The processor continuously tracks the user's spatial coordinates via Bluetooth-enabled angular antenna array and maps these coordinates to the camera's view coordinate system. When the user's spatial coordinates fall within the virtual fence area, the processor determines that the user has entered the street photography node's shooting area and then generates a trigger signal.

[0055] After the street photography node is triggered, the camera at the sensing and acquisition node captures consecutive image frames before and after the current moment. Since the camera always captures images at a fixed sampling rate and writes them to the circular buffer, when the trigger signal is generated, the processor can directly extract multiple frames of images before and after the trigger moment from the circular buffer, and select the frame or multiple frames with the best posture (e.g., complete face, relaxed limbs) and the highest clarity (e.g., maximum image entropy value, strongest edge gradient) as the capture result.

[0056] The generation strategy for street photography nodes is configured as follows: determine whether the user is an authorized user. If authorized, the captured image is directly pushed to the user's terminal device as the output image; if not authorized, the captured image is marked as a temporary cached state and deleted or blurred after the preset lifespan expires.

[0057] Scene Node 2: Blind Box Gacha Node A blind box drawing node is a scene node used to artistically redraw the user's image in a random style after the user triggers an interaction.

[0058] In this embodiment, the blind box card-drawing node is equipped with one or more sensors, including infrared sensors, millimeter-wave radar, or pressure sensors. These sensors are used to detect the user's approach or interaction. For example, an infrared sensor can detect the user's hand entering the sensing area, millimeter-wave radar can detect the user's movement trajectory, and a pressure sensor can detect the user's stepping or touching actions.

[0059] The trigger signal for the blind box card-drawing node is generated by the scene node when the sensor detects a trigger event. When a user enters the interactive area of ​​the blind box card-drawing node and triggers the sensor, such as by waving their hand through the infrared sensing area or stepping on the pressure pad, the scene node generates a trigger signal.

[0060] The generation strategy configuration for blind box gacha nodes includes a style card pool. The style card pool stores multiple style entries, each associated with a hint template, model parameters, and a cooldown period. Style entries include, for example, cyberpunk, oil painting, watercolor, and anime styles. The cooldown period limits the probability of the same style being drawn repeatedly within a short period, increasing the diversity of the output.

[0061] After the blind box card-drawing node is triggered, the cloud server extracts the image at the corresponding moment from the image cache based on the trigger signal, and then performs a style sampling process: First, a random seed is generated using a hash algorithm or pseudo-random number generation algorithm, taking the session token, device token, and current timestamp as input. The session token is a unique identifier for the user's current session, the device token is a unique identifier for the scene node, and the current timestamp ensures randomness in triggering at different times. These three elements work together as factors in generating the random seed, allowing the same style decision results to be reproduced within the same session, facilitating user traceability and operational auditing. Different sessions, due to their different timestamps, will receive differentiated style outputs.

[0062] Secondly, the sampling probability of each style item is calculated based on its base weight and cooldown count in the style card pool. The base weight reflects the style's tendency to be selected under default conditions, while the cooldown count reflects the frequency of its recent selection. The more times a style has been selected, the lower its probability of being selected again, thus ensuring style diversity.

[0063] Then, random sampling is performed using the generated random seed within the above sampling probability distribution to determine the target style.

[0064] Finally, the image is stylized and redrawn according to the cue template and model parameters corresponding to the target style. Specifically, the cue template of the target style and the user image are input into an AIGC model such as the Stable Diffusion model. The model uses the user image as the base map and the cue template as the guide, and redraws the background and overall image style while keeping the user's facial features basically unchanged, to obtain a stylized output image.

[0065] Scene Node 3: Advertising Interaction Node An interactive ad node is a scene node used to generate synthetic images of user interaction with ad content as the user views an ad screen.

[0066] In this embodiment, the advertising interaction node includes a smart screen node and associated sensing and acquisition nodes. The smart screen node is equipped with a display panel and a Bluetooth angle-of-arrival antenna array for playing advertising materials and sensing the location of nearby users. The advertising materials are pre-produced multimedia content that includes one or more predetermined keyframes during playback. Keyframes are the most visually impactful or brand-recognizable moments in the advertisement, such as the moment when the brand logo appears in full, or the climax of a close-up product shot.

[0067] The trigger signal for the advertising interaction node is generated by the smart screen node when the advertising material reaches a predetermined keyframe. Specifically, while playing the advertising material, the smart screen node's player maintains the current playback progress. When the playback progress reaches the preset keyframe time point, the smart screen node generates a trigger signal. However, not all users passing by the advertising screen will trigger a capture; triggering also requires meeting spatial conditions: the smart screen node's Bluetooth angle of arrival antenna array continuously receives Bluetooth beacon frames broadcast by the user's identity node, calculates the user's angle of arrival relative to the smart screen node based on the phase difference of the beacon frames arriving at each antenna element, and calculates the distance value based on the signal strength. The smart screen node only sends a trigger signal when the user's spatial coordinates are within the predetermined range of the smart screen node (e.g., less than 2 meters away) and facing the screen. This dual condition ensures that capturing occurs only when the user is truly paying attention to the advertising content, avoiding invalid snapshots.

[0068] In one optional implementation of this embodiment, before the smart screen node sends a trigger signal, the sensing and acquisition node also performs visual verification on the currently cached image. Visual verification includes two detections: The first detection step is to determine whether the user is facing the smart screen node in the current cached image. The processor estimates the user's face pose in the image and calculates the angle between the user's facial orientation and the screen's normal direction. If the angle is less than a predetermined threshold, such as 30 degrees, the user is determined to be facing the screen.

[0069] The second detection step involves checking whether the currently cached image contains the content of the advertising material. The processor performs image recognition on the screen area in the image to determine whether the content displayed in the screen area matches the currently playing advertising material.

[0070] The smart screen node is only allowed to send a trigger signal when both checks pass. This visual verification mechanism effectively avoids invalid captures triggered simply because the user is passing by the screen, improving the accuracy and effectiveness of the capture.

[0071] After the advertising interaction node is triggered, the sensing and acquisition node extracts the image at the corresponding moment from the image cache based on the timestamp carried by the trigger signal, and sends the image to the cloud server.

[0072] The cloud server retrieves the metadata (Ad_Metadata) of the current ad creative. The metadata of the ad creative is structured data pre-configured before ad delivery, including but not limited to: brand identity, product category, screen display area location (such as the coordinate range of the ad content display area in the image coordinate system), keyframe images, a list of virtual props that can be generated, a Prompt template (such as a user holding {product} interacting with the {brand} ad in the background screen), and security restrictions (such as prohibited areas and content).

[0073] The cloud server performs human face segmentation on the received images, using deep learning-based semantic segmentation models such as U-Net or Self-Attention, to generate a human face mask—a pixel mask representing the human body regions in the image. The human face mask is used to identify which pixels in the image belong to the user's body and which belong to the background.

[0074] The cloud server determines the screen mask based on the location of the displayed area. The area in the image that lies outside both the portrait mask and the screen mask is designated as the editable area. This area forms the background space in the image, where virtual props or lighting effects can be safely added without covering the user's body or obscuring the actual content displayed on the screen.

[0075] The cloud server fills the brand logo and product category into the prompt template, for example, filling the {product} slot in "A user is holding {product}", generating complete prompt text. Then, it redraws the image within the editable area according to the prompt text. Specifically, the cloud server calls an AIGC model, such as the Stable Diffusion model combined with ControlNet pose constraints, to generate a virtual object containing the brand logo within the editable area, using the original image as the base map and the prompt text as the guide. For example, it generates a virtual Coke bottle at the user's hand position and renders the color, lighting, and shadow of the virtual object to ensure consistency with the lighting direction and environment of the original image. After generation, the output image is obtained, which is a composite image of the user interacting with the advertising content.

[0076] Scene Node 4: Official Photography Node A designated photography node is a scene node used to perform personalized shooting in response to user voice commands.

[0077] In this embodiment, the trigger signal for the designated camera node is generated when the scene node establishes a connection with the user identity node via Near Field Communication (NFC). The user brings their personal identity node (in card form) close to the NFC reader of the designated camera node (at a distance of less than 4 cm). The NFC reader reads the user identifier stored in the user identity node, completes identity authentication, establishes a connection, and then generates a trigger signal.

[0078] Once the designated photography node is triggered, the scene node receives voice commands sent by the user through a terminal device such as a mobile phone. The user can run the "AI Photographer" intelligent agent application on their mobile phone and speak their shooting requirements, such as "Make me look thinner and blur the background."

[0079] The scene node's processor performs Automatic Speech Recognition (ASR) and Natural Language Understanding (NLU) on the voice commands, parsing the user's natural language description into structured control parameters. These control parameters include: pan / tilt / zoom control parameters (horizontal rotation, pitch rotation, and zoom parameters) and image processing parameters such as beautification intensity, background blur level, and exposure compensation value.

[0080] The processor adjusts the gimbal attitude of the cameras at the sensing and acquisition nodes, such as horizontal rotation angle, pitch rotation angle, and zoom ratio, according to the gimbal control parameters, so that the cameras are aimed at the user and obtain the best composition. At the same time, the processor performs image optimization processing on the acquired images according to the image processing parameters, such as portrait beautification, background blurring, and color correction, to obtain the output image.

[0081] Finally, the scene node establishes a link between the output image and the user identifier, recording the output image under that user's name. It's important to note that all data generated during the exclusive photography node's session validity period strictly belongs to that user identifier; other users cannot retrieve or access it, ensuring data privacy.

[0082] Scene Node 5: Fixed-Point Check-in Node A fixed-point check-in node is a scene node used to automatically trigger shooting when a user makes a specific gesture or movement.

[0083] In this embodiment, the trigger signal for the fixed-point check-in node is generated when the camera of the sensing and acquisition node recognizes the user's predetermined action. The camera of the sensing and acquisition node continuously acquires image sequences of the predetermined area, and the processor performs pose recognition on the image sequences, such as using a deep learning-based pose estimation model like OpenPose or MediaPipe, to extract the sequence of key points of the human skeleton.

[0084] The processor predefines one or more predefined action templates, such as "holding a V sign for 2 seconds," "making a heart shape with both hands," and "jumping." Each predefined action template is defined by a series of spatial positions and temporal variation patterns of key points on the human skeleton. When the processor recognizes that the current user's posture matches a predefined action template, if the deviation of the key point positions is less than a predetermined threshold and the duration meets the requirements, a trigger signal is generated.

[0085] Once the fixed-point check-in node is triggered, the camera at the sensing and acquisition node extracts images before and after the trigger time from the circular buffer as the capture result, and pushes the captured image directly to the user as the output image.

[0086] Example 4 This embodiment provides a method for cross-node image asset aggregation and unified delivery.

[0087] In this embodiment, the cloud server generates an asset record (Asset_Record) for each media file generated for each scene node, including the original captured image and the AI-generated image. The asset record is a structured data entry that includes at least the following fields: Asset_ID (file identifier, used to uniquely identify the media file); User_UUID (user identifier, a unique identifier of the user to whom the file belongs); Scene_Type (scene node type that generated the file); Capture_Time (time stamp of the file generation); Spatial coordinates (user's spatial location when the file was generated); Generation_Mode (generation method, such as "original capture", "AI redraw", "style transfer", etc.); and file storage path.

[0088] All asset records are stored in a structured manner in an asset index table on a cloud server. This asset index table is global and not limited to any single scene node.

[0089] When a user initiates an asset query request through a terminal device such as a mobile app, for example through an NFC "tap-to-find" scenario node or through the "My Album" function in the app, the cloud server uses the user's user identifier as the sole query condition to search the asset index table and retrieve all asset records associated with that user identifier.

[0090] The cloud server sorts the recalled asset records by shooting time, generating a spatiotemporal image sequence for the user. This is a complete list of all image assets generated by the user at various scene nodes throughout the entire space, arranged in chronological order. Users can browse all the images they generated at different times, locations, and scene nodes on their terminal devices according to the timeline.

[0091] Finally, the cloud server pushes the image files corresponding to the sorted asset records to the target user's terminal device. Users can obtain a complete image sequence across all scene nodes with a single query on their terminal device, eliminating the need to access or download each scene node separately, significantly improving the user experience.

[0092] Example 5 This embodiment provides a specific implementation method for time synchronization and precise snapshot in an interactive advertising scenario.

[0093] In interactive advertising scenarios, there is a time difference between the moment a user interacts with the ad and the moment the camera captures the corresponding image. This time difference stems from several factors: the transmission delay of the trigger signal (network transmission time from the smart screen node to the sensing and acquisition node), the camera's shutter delay (time from receiving the trigger command to completing the exposure), and the clock deviation between the two nodes. Without compensation for these delays, the captured image will not match the ad's keyframe; for example, the ad's keyframe may have ended before the camera completes exposure, or the camera may expose prematurely before the ad has reached the keyframe.

[0094] To address this issue, in this embodiment, the sensing and acquisition nodes and the smart screen nodes maintain clock synchronization via heartbeat packets. Specifically, the two nodes send heartbeat messages to each other at a predetermined period, such as every second, with each heartbeat message carrying the sender's current timestamp. The receiver records the arrival time of the heartbeat message and calculates the round-trip time (RTT) by combining it with the sending timestamp. Through multiple consecutive heartbeat measurements, the nodes can estimate statistical values, such as moving averages, of the clock offset and network latency between the two nodes.

[0095] When a smart screen node sends a trigger signal, it includes a precise current timestamp. Upon receiving the trigger signal, the sensing and acquisition node reads the timestamp and, based on previously estimated clock offsets and network transmission delays, converts the timestamp to the equivalent time under the sensing and acquisition node's local clock. The specific calculation method is as follows: Equivalent time = trigger signal timestamp + clock skew + network transmission delay.

[0096] Based on the calculated equivalent time, the sensing and acquisition nodes locate all image frames acquired before and after a given moment in the circular buffer, including all frames within a 0.5-second range before and after that moment. From these, the frame with the best pose and highest clarity is selected as the captured image. Through this clock synchronization and delay compensation mechanism, the time alignment accuracy between the captured image and the advertising keyframe can reach the millisecond level.

[0097] In one optional implementation of this embodiment, the smart screen node also sends a pre-triggered message (PRE_TRIGGER) 200 milliseconds before the keyframe arrives. After receiving the pre-triggered message, the sensing and acquisition node performs focus lock and exposure lock operations in advance, so that the camera is already in the best shooting state when the keyframe actually arrives, further shortening the response time from triggering to completing the exposure.

[0098] In another optional implementation of this embodiment, the trigger signal sent by the smart screen node also carries an incrementing sequence number. Each time the sensing and acquisition node receives a trigger signal, it records its sequence number. If the sequence number of a newly received trigger signal does not meet the continuity condition with the sequence number of the previously received trigger signal (e.g., the sequence number changes or repeats), the current trigger signal is discarded. This mechanism effectively prevents duplicate or erroneous triggering caused by network retransmissions or out-of-order delivery.

[0099] Example 6 This embodiment provides a specific implementation method for AIGC image generation and consistency verification.

[0100] In this embodiment, the AIGC engine of the cloud server uses a diffusion model as the basic architecture for image generation. The diffusion model is a generation model based on progressive denoising; its basic idea is to start with pure Gaussian noise and gradually recover a clear image through a series of denoising steps. Stable Diffusion and Flux are commonly used diffusion model implementations in this field.

[0101] During image redrawing, the cloud server first performs portrait segmentation on the original image to generate a portrait mask. The portrait mask is a binary image where pixels belonging to the human body have a value of 1, and pixels belonging to the background have a value of 0. The purpose of the portrait mask is to identify which areas in the image belong to the user's body; these areas should remain unchanged in subsequent generation processes to maintain user identity consistency.

[0102] The cloud server determines the editable area based on the scene type identifier. In an interactive advertising scenario, the editable area is the region in the image that is outside the portrait mask and outside the screen mask; in a blind box gacha scenario, the editable area is the entire background area in the image that is outside the portrait mask. Determining the editable area ensures that the generation operation does not cover or destroy the user's original facial features and body contours.

[0103] The cloud server loads the corresponding adaptation model (LoRA model) according to the generation strategy configuration. LoRA (Low-Rank Adaptation) is a lightweight model fine-tuning technique that adds a small number of trainable parameters to a pre-trained model to achieve rapid adaptation for specific styles, brands, or categories. For example, a LoRA model pre-trained for the Coca-Cola brand can make the generated virtual Coca-Cola bottles more similar to the real product in shape, color, and lighting effects.

[0104] The cloud server inputs the raw imagery into a pose estimation network such as OpenPose or MediaPipe to generate a pose constraint map. A pose constraint map is a skeleton image composed of lines connecting key points of the human skeleton, encoding the user's current body pose and hand position. During the generation process of the diffusion model, the pose constraint map serves as the control condition input to ControlNet, constraining the position and pose of virtual objects in the generated result to match the user's actual pose. For example, if the user's hand is on the right side of the body, the generated virtual Coke bottle should also appear near the right hand, not on the left hand or away from the body.

[0105] The cloud server uses the original image as a base map and the pose constraint map as control conditions to perform diffusion generation within the editable area using a LoRA-adapted model. During the generation process, the model only adds or modifies pixels within the editable area, while the areas covered by the human face mask (user's body) and the areas covered by the screen mask (screen display content) remain unchanged.

[0106] After generation, the cloud server performs a consistency check on the output image. The consistency check includes one or more of the following checks: FaceID similarity detection: Calculates the feature similarity between the user's facial region in the output image and the user's facial region in the original image. This can be achieved by using face recognition models such as FaceNet or ArcFace to extract facial feature vectors and calculating cosine similarity. If the similarity is below a predetermined threshold, such as 0.8, it indicates that the generation process excessively modified the user's facial features, potentially causing user identity distortion. In this case, the output image is discarded.

[0107] Advertising element detection: Detects whether the output image contains expected advertising elements such as brand logos or virtual products. If not detected, it means the generation process failed to successfully add virtual objects, and the output image is discarded.

[0108] Screen content consistency detection: Detects whether the content displayed in the screen area of ​​the output image is consistent with the content displayed in the screen area of ​​the original image, ensuring that the actual advertising content played on the screen is not mistakenly modified during the generation process.

[0109] Lighting direction consistency detection: This detects whether the lighting direction of virtual objects in the output image is consistent with the actual lighting direction of the scene in the original image. Specifically, the processor extracts the scene's lighting direction information from the original image, such as by analyzing shadow directions and surface brightness gradients, and then constrains the lighting effects of the virtual objects to match this lighting direction during the generation process.

[0110] If any of the above verifications fails, the cloud server re-executes the generation process using the same context data packet. If the maximum number of consecutive retries is reached (e.g., 3 times) and the verification still fails, the generated result is discarded, and only the original authorized photo is retained as a backup output. This multi-verification mechanism ensures the quality and identity consistency of the output image.

[0111] Although the invention has been specifically shown and described in conjunction with preferred embodiments, those skilled in the art should understand that various changes in form and detail may be made to the invention without departing from the spirit and scope of the invention as defined in the appended claims, all of which shall be within the scope of protection of the invention.

Claims

1. A distributed scene-based intelligent shooting method, characterized in that, Includes the following steps: The user's re-identification feature vector is obtained through the cameras at the sensing and acquisition nodes; Users broadcast Bluetooth beacon frames through a user identity node they carry with them, and receive the Bluetooth beacon frames through the Bluetooth scanning module of the sensing and acquisition node, extracting the user identifier stored in the user identity node; The re-identification feature vector is associated and bound with the user identifier; When the camera recognizes that a bound user has arrived at a predetermined scene node, the scene node sends a shooting trigger signal, which carries the type identifier of the scene node; Based on the type identifier, capture and generate contextualized images for user interaction.

2. The distributed scene-based intelligent shooting method according to claim 1, characterized in that, The step of associating and binding the re-identification feature vector with the user identifier includes: The re-identification feature vector is compared with the registration feature vector in the authorized user database; If the comparison matches, the user corresponding to the re-identified feature vector is marked as an authorized user, and the user identifier is bound to the authorized user; If the comparison is inconsistent, the user corresponding to the re-identification feature vector will be marked as an unregistered user, and the corresponding image data will be marked as a temporary cache state, and deleted or blurred after the preset life cycle is reached.

3. The distributed scene-based intelligent shooting method according to claim 1, characterized in that, After the user identity node broadcasts the Bluetooth beacon frame, it also includes: The sensing and acquisition node receives the Bluetooth beacon frame via its Bluetooth angle-of-arrival antenna array, calculates the angle of arrival of the user relative to the sensing and acquisition node based on the phase difference of the Bluetooth beacon frame arriving at each antenna element, calculates the distance value based on the signal strength, and determines the user's spatial coordinates using the angle of arrival and the distance value.

4. The distributed scene-based intelligent shooting method according to claim 1, characterized in that, The scene nodes include one or more of the following: The trigger signal for the street photography node is generated by the camera of the sensing and acquisition node when it recognizes that the bound user has entered the predetermined framing range; The trigger signal for the blind box card drawing node is generated by the sensor configured in the scene node when a trigger event is detected; The corresponding trigger signal for the advertising interaction node is generated by the smart screen node when the advertising material is played to the predetermined key frame; The trigger signal for the designated photography node is generated when the scene node establishes a connection with the user identity node through near-field communication. The trigger signal for the fixed check-in node is generated when the camera of the sensing and acquisition node recognizes the user's predetermined action.

5. The distributed scene-based intelligent shooting method according to claim 4, characterized in that, Before the advertising interaction node sends the trigger signal, it also includes: The sensing and acquisition node performs visual verification on the current cached image. The visual verification includes detecting whether the user is facing the smart screen node and detecting whether the current cached image contains the display content of the advertising material. The trigger signal is allowed to be sent only if both tests pass.

6. The distributed scene-based intelligent shooting method according to claim 4, characterized in that, When the scene node is an advertising interaction node, the step of capturing and generating a scene-based image for user interaction based on the type identifier includes: The sensing and acquisition node extracts the image at the corresponding moment from the image cache according to the timestamp carried by the trigger signal, and sends the image to the cloud server; The cloud server obtains the metadata of the advertising material, which includes brand identity and product category; The cloud server performs portrait segmentation on the image to determine the editable area, fills the brand logo and product category into the prompt template, and performs image redrawing within the editable area according to the prompt template to generate a virtual object containing the brand logo, thereby obtaining the output image.

7. The distributed scene-based intelligent shooting method according to claim 4, characterized in that, When the scene node is a blind box card-drawing node, the step of capturing and generating a scene-based image for user interaction based on the type identifier includes: The image at the corresponding moment is extracted from the image buffer according to the trigger signal; A random seed is generated using the session identifier, device identifier, and current timestamp; The sampling probability of each style entry is calculated based on the weight and cooldown number of each style entry in the style card pool, and the target style is determined by sampling in the sampling probability distribution using the random seed. The image is stylized and redrawn according to the target style to obtain the output image.

8. The distributed scene-based intelligent shooting method according to claim 4, characterized in that, When the scene node is a street photography node, the trigger signal is generated by the camera of the sensing and acquisition node in the following manner: Define a virtual fence area in the view coordinate system of the camera; The user's spatial coordinates are determined via a Bluetooth angle-of-arrival antenna array; The trigger signal is generated when the user's spatial coordinates fall within the virtual fence area.

9. The distributed scene-based intelligent shooting method according to claim 4, characterized in that, When the scene node is a designated photography node, the step of capturing and generating a scene-based image for user interaction based on the type identifier includes: The scene node acquires the voice commands sent by the user through the terminal device; The voice commands are parsed into gimbal control parameters and image processing parameters; The camera orientation of the sensing and acquisition node is adjusted according to the gimbal control parameters, and the acquired images are optimized according to the image processing parameters to obtain the output image. Establish the association between the output image and the user identifier.

10. The distributed scene-based intelligent shooting method according to claim 1, characterized in that, After capturing and generating contextualized images for user interaction, the process also includes: Using the user identifier as an index, all captured and generated images associated with the target user are retrieved; The recalled images are sorted according to the shooting time to generate a spatiotemporal image sequence of the target user; The spatiotemporal image sequence is pushed to the target user's terminal device.

Images