An object search method, system, device, electronic equipment and storage medium

Abstract

The embodiment of the disclosure provides a kind of object search method, system, device, electronic equipment and storage medium, involve cloud side collaborative technical field, in which, a kind of object search method, applied to edge node includes: in response to the specified command issued by cloud side processing system, the video data collected by target monitoring device is acquired;From the picture of video data containing candidate object, the object area of candidate object is intercepted, to obtain specified picture;The specified picture is sent to cloud side processing system, to make cloud side processing system match the object in specified picture and the object in target picture, to obtain analysis result;If analysis result indicates matching, the indication information indicating that the object in specified picture is matched successfully is fed back to edge node;After receiving the indication information, based on the picture containing candidate object in video data, the search result corresponding to target picture is fed back to requester.It can be seen that the above method can reduce the consumption of network bandwidth.

Description

Technical Field

[0001] This disclosure relates to the field of cloud-edge collaboration technology, and in particular to an object search method, system, apparatus, electronic device, and storage medium. Background Technology

[0002] Currently, smart cities, smart parks, and other key locations and roads are equipped with a large number of surveillance devices to meet increasingly demanding security requirements. In surveillance scenarios, searching for objects, such as specific individuals, has become a crucial security function. Existing technologies typically employ object search systems comprised of multiple nodes on the cloud side. When a cloud-side node receives a search task for a specific object, it acquires video data collected by various surveillance devices and analyzes the acquired video segments to locate the object.

[0003] However, the large amount of video data collected by each monitoring device needs to be uploaded to the cloud-side nodes, which consumes a lot of network bandwidth resources. Summary of the Invention

[0004] The purpose of this disclosure is to provide an object search method, system, apparatus, electronic device, and storage medium to solve the problem of consuming large amounts of network bandwidth resources. The specific technical solution is as follows:

[0005] In a first aspect, embodiments of this disclosure provide an object search method applied to edge nodes, the method comprising:

[0006] In response to a specified command issued by the cloud-side processing system, video data collected by the target monitoring device is acquired; wherein, the specified command is issued by the cloud-side processing system to the edge node after acquiring a target image containing the object to be searched sent by the requester; and candidate objects existing in the video data are determined.

[0007] From the images containing the candidate objects in the video data, the object region of the candidate object is extracted to obtain the specified image;

[0008] The specified image is sent to the cloud-side processing system so that the cloud-side processing system can perform matching analysis on the objects in the specified image and the objects in the target image to obtain the analysis result; if the analysis result shows that they match, the edge node is fed back an indication message indicating that the objects in the specified image have been successfully matched.

[0009] Upon receiving the instruction information, based on the image of the candidate object contained in the video data, the search results corresponding to the target image are fed back to the requesting party.

[0010] Optionally, determining the candidate objects present in the video data includes:

[0011] Using a designated business module, candidate objects existing in the video data are determined; wherein, the designated business module is a module that generates a container-like structure by running a target image used to determine the candidate objects.

[0012] Optionally, before determining the candidate objects present in the video data using a designated business module, the method further includes:

[0013] If the specified business module is not detected, the target image is obtained from the image library of the cloud-side processing system;

[0014] Run the target image to generate a container-shaped module, which will serve as the specified business module.

[0015] Optionally, determining the candidate objects present in the video data using a designated business module includes:

[0016] The specified business module is invoked to perform object location detection on the video data. Using the object location detection results, the trajectory of the object in the video data is generated, and the object indicated by the trajectory is identified as a candidate object in the video data.

[0017] Secondly, embodiments of this disclosure provide an object search method applied to a cloud-side processing system, the method comprising:

[0018] Obtain the target image containing the object to be searched sent by the requester, and send the specified command to the edge node;

[0019] Obtain a specified image sent by an edge node; wherein the specified image is generated by the edge node in response to a specified command issued by the cloud-side processing system, and uploaded to the cloud-side processing system according to a specified generation method, the specified generation method including obtaining video data collected by the target monitoring device; determining candidate objects existing in the video data; and extracting the object region of the candidate object from the image containing the candidate object in the video data to obtain the specified image;

[0020] The objects in the target image and the objects in the specified image are matched and analyzed to obtain the analysis results;

[0021] If the analysis results indicate a match, the edge node sends back an indication that the object in the specified image has been successfully matched. Upon receiving the indication, the edge node, based on the image containing the candidate object in the video data, sends back the search result corresponding to the target image to the requester.

[0022] Optionally, a matching analysis is performed on the objects in the target image and the objects in the specified image to obtain the analysis results, including:

[0023] By using a pedestrian re-identification algorithm, the algorithm analyzes whether the objects in the specified image match the objects in the target image, and obtains the analysis results.

[0024] Optionally, the number of edge nodes can be multiple;

[0025] The process of issuing specified commands to edge nodes includes:

[0026] According to the search assistance information corresponding to the target image, an edge node to be utilized is selected from a plurality of edge nodes; wherein, the search assistance information includes information characterizing the search area corresponding to the target image;

[0027] Send the specified command to the selected edge node.

[0028] Thirdly, embodiments of this disclosure provide an object search system, the system including a cloud-side processing system and edge nodes;

[0029] The cloud-side processing system is used to send a specified command to the edge node after obtaining the target image containing the object to be searched sent by the requester.

[0030] The edge node is used to respond to the specified command, acquire video data collected by the target monitoring device; determine candidate objects existing in the video data; extract the object region of the candidate object from the image containing the candidate object in the video data to obtain a specified image, and send the specified image to the cloud-side processing system;

[0031] The cloud-side processing system is also used to acquire the specified image sent by the edge node; perform matching analysis on the objects in the target image and the objects in the specified image to obtain the analysis result; if the analysis result shows a match, it feeds back to the edge node an indication that the objects in the specified image have been successfully matched;

[0032] The edge node is also configured to, upon receiving the indication information, provide the requester with the search results corresponding to the target image based on the image of the candidate object contained in the video data.

[0033] Fourthly, embodiments of this disclosure provide an object search device applied to edge nodes, the device comprising:

[0034] The first acquisition module is used to acquire video data collected by the target monitoring device in response to a specified command issued by the cloud-side processing system; wherein, the specified command is issued by the cloud-side processing system to the edge node after acquiring the target image containing the object to be searched sent by the requester;

[0035] The determination module is used to determine the candidate objects present in the video data;

[0036] The cropping module is used to crop the object region of the candidate object from the image containing the candidate object in the video data to obtain a specified image;

[0037] The sending module is used to send the specified image to the cloud-side processing system, so that the cloud-side processing system can perform matching analysis on the objects in the specified image and the objects in the target image to obtain the analysis result; if the analysis result shows that they match, it feeds back to the edge node an indication that the objects in the specified image have been successfully matched.

[0038] The first feedback module is used, after receiving the instruction information, to provide the requester with the search results corresponding to the target image based on the image of the candidate object contained in the video data.

[0039] Fifthly, embodiments of this disclosure provide an object search device applied to a cloud-side processing system, the device comprising:

[0040] The second acquisition module is used to acquire the target image containing the object to be searched sent by the requester and send the specified command to the edge node;

[0041] The third acquisition module is used to acquire a specified image sent by the edge node; wherein the specified image is generated by the edge node in response to a specified command issued by the cloud-side processing system, and uploaded to the cloud-side processing system according to a specified generation method, the specified generation method including acquiring video data collected by the target monitoring device; determining candidate objects existing in the video data; and extracting the object region of the candidate object from the image containing the candidate object in the video data to obtain the specified image;

[0042] The analysis module is used to perform matching analysis on objects in the target image and objects in the specified image to obtain analysis results;

[0043] The second feedback module is used to provide feedback to the edge node if the analysis result shows a match, indicating that the object in the specified image has been successfully matched. This allows the edge node to provide feedback to the requester with the search results corresponding to the target image based on the image containing the candidate object in the video data upon receiving the feedback information.

[0044] In a sixth aspect, embodiments of this disclosure provide an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0045] Memory, used to store computer programs;

[0046] A processor, used to execute a program stored in memory, implements the method steps for searching any object.

[0047] In a seventh aspect, embodiments of this disclosure provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of an arbitrary object search method.

[0048] Beneficial effects of the embodiments disclosed herein:

[0049] This disclosure provides an object search method. An edge node responds to a specified command issued by a cloud-based processing system, acquires video data collected by a target monitoring device, identifies candidate objects within the video data, extracts the object region of the candidate object from an image containing the candidate object in the video data, obtains a specified image, and sends the specified image to the cloud-based processing system. The cloud-based processing system acquires the target image, performs matching analysis on the objects in the target image and the objects in the specified image, and obtains analysis results. If the analysis results indicate a match, it sends feedback to the edge node indicating a successful match of the object in the specified image. Upon receiving the feedback, the edge node, based on the image containing the candidate object in the video data, sends feedback to the requesting party the search result corresponding to the target image.

[0050] In summary, in this embodiment, the object search method is implemented through cloud-edge collaboration. This eliminates the need to upload video data from the monitoring device to the cloud-side processing system; instead, the data is uploaded to edge nodes. These edge nodes then perform video analysis and upload a small, specified image to the cloud-side processing system. The cloud-side processing system then matches and analyzes the specified image against the target image. Simultaneously, the search results are directly fed back to the requesting party through the edge nodes. Therefore, compared to existing technologies, this solution, by implementing the object search method through cloud-edge collaboration, can reduce network bandwidth consumption.

[0051] In addition, by extracting the object region of the candidate object from the image containing the candidate object in the video data, the network bandwidth consumption is reduced compared to directly uploading the video data, while also protecting the privacy of the monitoring site to a certain extent. Furthermore, compared to the centralized deployment of object search inference services on the cloud side, which leads to extremely high performance requirements for the cloud side system, this solution can adopt a cloud-edge collaborative architecture, thus making it easier to achieve system expansion and greatly reducing the limitations on system scalability.

[0052] Of course, implementing any product or method of this disclosure does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other embodiments can be obtained based on these accompanying drawings.

[0054] Figure 1 A flowchart illustrating an object search method applied to edge nodes provided in an embodiment of this disclosure;

[0055] Figure 2 A flowchart illustrating an object search method applied to a cloud-side processing system, provided in an embodiment of this disclosure;

[0056] Figure 3(a) is a schematic diagram of the structure of an object search system provided in an embodiment of this disclosure;

[0057] Figure 3(b) is a schematic diagram of the structure of an object search system provided in an embodiment of this disclosure;

[0058] Figure 4 This is another structural schematic diagram of an object search system provided in an embodiment of the present disclosure;

[0059] Figure 5 A flowchart illustrating an object search process from the perspective of user interaction with an object search system, provided as an embodiment of this disclosure;

[0060] Figure 6 A schematic diagram illustrating the working principle of an object search system provided in this embodiment of the disclosure;

[0061] Figure 7 A schematic diagram illustrating the working principle of another object search system provided in this embodiment of the disclosure;

[0062] Figure 8 This is a schematic diagram illustrating the working principle of a pedestrian detection module provided in an embodiment of the present disclosure;

[0063] Figure 9 This is a schematic diagram of the structure of an object search device applied to an edge node, provided in an embodiment of the present disclosure;

[0064] Figure 10 A schematic diagram of the structure of an object search device applied to a cloud-side processing system provided in this embodiment of the present disclosure;

[0065] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0066] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art based on this disclosure are within the scope of protection of this disclosure.

[0067] With societal development, object search systems have become crucial in the security field. Even after implementing extensive video surveillance at key locations and along roads, mainstream solutions still require uploading the surveillance video to the cloud. Image and video analytics technologies based on artificial intelligence algorithms have flourished in the past decade, and thanks to the elasticity and massive computing power provided by cloud computing, deploying and running video analytics applications in the cloud has become a popular technical solution. In existing technologies, the entire object search system is deployed in the cloud, and video data from monitoring devices needs to be uploaded to the cloud, undoubtedly consuming significant network bandwidth. Furthermore, because the entire object search system is deployed in the cloud, video data containing sensitive information is transmitted across the public network, posing a privacy risk. Moreover, the cloud centrally handles the entire analysis process of the monitoring data—that is, the AI ​​inference business is centrally deployed in the cloud—placing extremely high performance requirements on the cloud-based system used for object search, thus limiting the system's scalability.

[0068] Based on the above, firstly, in order to solve the problem of consuming a large amount of network bandwidth resources, from the perspective of edge nodes, this disclosure provides an object search method.

[0069] An edge node, as opposed to a cloud computing data center, refers to a network node with few intermediate links between it and the final access node. This disclosure does not limit the specific form of an edge node. For example, an edge node can be an edge gateway, a home gateway, an IoT (Internet of Things) gateway, or other similar devices. It should be noted that in specific applications, an edge node can also be a monitoring device, such as a camera.

[0070] Furthermore, edge nodes are nodes within the object search system, which also includes a cloud-side processing system. The cloud-side processing system is deployed in the cloud, and object search can be achieved through the interaction between the edge nodes and the cloud-side processing system. Moreover, the cloud-side processing system can consist of one or more nodes. For a description of the specific device configuration of the cloud-side processing system, please refer to the embodiment content of the object processing system.

[0071] One of the object search methods provided by this disclosure from the perspective of edge nodes may include:

[0072] In response to a specified command issued by the cloud-side processing system, video data collected by the target monitoring device is acquired; wherein, the specified command is issued by the cloud-side processing system to the edge node after acquiring a target image containing the object to be searched sent by the requester; and candidate objects existing in the video data are determined.

[0073] From the images containing the candidate objects in the video data, the object region of the candidate object is extracted to obtain the specified image;

[0074] The specified image is sent to the cloud-side processing system so that the cloud-side processing system can perform matching analysis on the objects in the specified image and the objects in the target image to obtain the analysis result; if the analysis result shows that they match, the edge node is fed back an indication message indicating that the objects in the specified image have been successfully matched.

[0075] Upon receiving the instruction information, based on the image of the candidate object contained in the video data, the search results corresponding to the target image are fed back to the requesting party.

[0076] In summary, in this embodiment, the object search method is implemented through cloud-edge collaboration. This eliminates the need to upload video data from the monitoring device to the cloud-side processing system; instead, the data is uploaded to edge nodes. These edge nodes then perform video analysis and upload a small, specified image to the cloud-side processing system. The cloud-side processing system then matches and analyzes the specified image against the target image. Simultaneously, the search results are directly fed back to the requesting party through the edge nodes. Therefore, compared to existing technologies, this solution, by implementing the object search method through cloud-edge collaboration, can reduce network bandwidth consumption.

[0077] In addition, by extracting the object region of the candidate object from the image containing the candidate object in the video data, the network bandwidth consumption is reduced compared to directly uploading the video data, while also protecting the privacy of the monitoring site to a certain extent. Furthermore, compared to the centralized deployment of object search inference services on the cloud side, which leads to extremely high performance requirements for the cloud side system, this solution can adopt a cloud-edge collaborative architecture, thus making it easier to achieve system expansion and greatly reducing the limitations on system scalability.

[0078] The following will describe, with reference to the accompanying drawings, an object search method provided by edge nodes according to an embodiment of the present disclosure. For example... Figure 1 As shown, an object search method is applied to edge nodes, and the method may include:

[0079] S101, in response to a specified command issued by the cloud-side processing system, acquire video data collected by the target monitoring device; wherein, the specified command is issued by the cloud-side processing system to the edge node after acquiring a target image containing the object to be searched sent by the requester;

[0080] The requester can be a streaming media server or a client corresponding to a cloud-based processing system. This disclosure does not limit the specific form of the requester.

[0081] After receiving a target image containing the object to be searched from the requester, the cloud-side processing system can interact with edge nodes to perform object searches on the target image. Specifically, the cloud-side processing system can send a specified command to the edge nodes, which then respond to the command. It should be noted that the specified command is used to trigger the edge nodes to perform video analysis and report the specified image of the candidate object obtained from the analysis. The specified command may carry the address of the video source to be used for acquiring video data. In this case, the monitoring device using the video source address to be used as the access address is identified as the target monitoring device, and the edge node acquires the video data collected by the target monitoring device. It can also be understood that the specified command can simply be a trigger command. In this case, after receiving the specified command, the edge node, in order to perform video acquisition, can first determine the video source address to be used according to a pre-set determination method, and then identify the monitoring device using the video source address to be used as the access address as the target monitoring device. For example, the pre-defined determination method may include: requesting the video source address to be used from the cloud-side processing system; or determining the video source address to be used from the accessible video source addresses pre-recorded by the edge node, and so on.

[0082] It should be noted that video data is usually compressed during transmission. Therefore, after acquiring video data from the target monitoring device, the video data can be decoded. OpenCV can be used for video decoding; OpenCV is a cross-platform computer vision and machine learning software library released under the Apache 2.0 license. Furthermore, the video data acquired by the target monitoring device can be referred to as the source video; and, for example, the monitoring device can be a camera, but this disclosure does not limit the specific form of the monitoring device.

[0083] Furthermore, it's understandable that to achieve object search, edge nodes need to access not only the cloud-side processing system via Ethernet but also the target monitoring device. To access the target monitoring device, edge nodes can use gateway forwarding, intranet pass-through, or other methods to form a unified network with the target monitoring device, thereby obtaining video data from it. Alternatively, they can connect directly to the monitoring camera via the bus to obtain video data from the target monitoring device. Intranet pass-through, also known as NAT (Network Address Translation) pass-through, is used to ensure that data packets with a specific source IP address and port number are not blocked by the NAT device and are correctly routed to the intranet host.

[0084] S102, determine the candidate objects present in the video data;

[0085] After acquiring the video data, it can be analyzed and processed to obtain candidate objects present in the video data, which can then be used for subsequent matching analysis with objects in the target image. It can be understood that the candidate objects in the video data are the various objects appearing in the frame of the video data.

[0086] For example, in one implementation, determining the candidate objects present in the video data may include step A1:

[0087] Step A1: Using a designated business module, determine the candidate objects present in the video data;

[0088] The designated business module is a module that generates a container form by running a target image used to determine candidate objects.

[0089] It should be noted that the designated business module can be an AI pedestrian detection module used to detect candidate objects. For example, the designated business module can be YOLOX (Megvii's open-source high-performance detector) running an AI (Artificial Intelligence) model. By deploying the module in container form, the expansion or deletion of the business module can be made convenient and quick.

[0090] Optionally, before determining the candidate objects present in the video data using a designated service module, steps B1-B2 may be included:

[0091] Step B1: If the specified business module is not detected, the target image is obtained from the image library of the cloud-side processing system.

[0092] Step B2: Run the target image to generate a container-shaped module, which serves as the specified business module;

[0093] The cloud-side processing system's image library contains target images, which may include installation packages or installers for the AI ​​pedestrian detection module; this disclosure does not limit the specifics. By running the target image, container-like modules can be generated on edge nodes to obtain the specified business modules.

[0094] For example, if an edge node detects that the AI ​​pedestrian detection module does not exist, it obtains the installation package of the AI ​​pedestrian detection module from the image library of the cloud-side processing system, runs the installation package of the AI ​​pedestrian detection module, installs the AI ​​pedestrian detection module, and packages the AI ​​pedestrian detection module into a container to obtain the specified business module.

[0095] In another implementation, determining the candidate objects present in the video data using a designated business module may include step C1:

[0096] Step C1: Invoke the designated business module to perform object location detection on the video data, use the object location detection results to generate the trajectory of the object in the video data, and determine the object indicated by the trajectory as a candidate object in the video data.

[0097] It should be noted that the trajectory of the object in the video data can be generated using the ByteTrack algorithm (a tracking method based on the tracking-by-detection paradigm). By associating it with the object features of the previous frame or k frames, continuous calibration of the same object is ensured across consecutive video frames, thereby obtaining the object's complete trajectory. The object features can include information such as clothing, body shape, and hairstyle. Compared to simply reporting the detected object in the video data as a candidate object to the cloud processing system, this implementation avoids reporting duplicate images of the same object to the cloud processing system by using the object indicated by the trajectory as a candidate object in the video data. This reduces computational load and ensures the accuracy of search results.

[0098] For example, the edge node calls the AI ​​pedestrian detection module to detect the position of pedestrian A in the video data, identify and mark all the positions that pedestrian A has traveled in the video, and use the ByteTrack algorithm to associate the clothing of pedestrian A in each frame to ensure that pedestrian A is continuously marked in consecutive video frames, thereby obtaining the completed walking trajectory of pedestrian A, and using the pedestrian A indicated by this walking trajectory as a candidate object in the video data.

[0099] It should be noted that the specific implementation of determining the candidate objects present in the video data described above is merely an example and should not be construed as limiting the embodiments of this disclosure.

[0100] S103, extract the object region of the candidate object from the image containing the candidate object in the video data to obtain the specified image;

[0101] The object region of the candidate object can be the smallest bounding box containing the candidate object. For example, from an image containing candidate object pedestrian A in video data, the smallest bounding box of the region containing pedestrian A is extracted to obtain a specified image, which is then packaged and sent to the cloud-based processing system. It is understood that by cropping the object region of the candidate object using the smallest bounding box to obtain a specified photo, since the specified photo only contains the candidate object, the surrounding scene, pedestrians, etc., can be left un-cropped, thus effectively ensuring the privacy of the monitored scene.

[0102] S104, the specified image is sent to the cloud-side processing system so that the cloud-side processing system can perform matching analysis on the objects in the specified image and the objects in the target image to obtain the analysis result; if the analysis result shows that they match, the system sends back an indication message to the edge node indicating that the objects in the specified image have been successfully matched.

[0103] If the analysis results show a match, it indicates that the object in the target image and the object in the specified image are the same person. In this case, the cloud-side processing system can send back an indication message to the edge node indicating that the object in the specified image and the object in the target image have successfully matched. It is understood that the indication message can be an identification code indicating a successful match, and this disclosure does not specifically limit the indication message.

[0104] In this process, edge nodes send specified images to the cloud-side processing system, which then analyzes them. For clarity and ease of understanding, this part is described in another embodiment.

[0105] S105, after receiving the instruction information, based on the image of the candidate object contained in the video data, the search results corresponding to the target image are fed back to the requesting party;

[0106] In addition, the search results for the target image can be video data of the candidate object's trajectory or an image containing the candidate object; this disclosure does not limit this.

[0107] For example, after receiving the knowledge information of a successful match, the edge node, based on the image of the candidate pedestrian A contained in the video data, sends back to the streaming media server a trajectory video representing the successful match containing the candidate pedestrian A.

[0108] In summary, in this embodiment, the object search method is implemented through cloud-edge collaboration. This eliminates the need to upload video data from the monitoring device to the cloud-side processing system; instead, the data is uploaded to edge nodes. These edge nodes then perform video analysis and upload a small, specified image to the cloud-side processing system. The cloud-side processing system then matches and analyzes the specified image against the target image. Simultaneously, the search results are directly fed back to the requesting party through the edge nodes. Therefore, compared to existing technologies, this solution, by implementing the object search method through cloud-edge collaboration, can reduce network bandwidth consumption.

[0109] In addition, by extracting the object region of the candidate object from the image containing the candidate object in the video data, the network bandwidth consumption is reduced compared to directly uploading the video data, while also protecting the privacy of the monitoring site to a certain extent. Furthermore, compared to the centralized deployment of object search inference services on the cloud side, which leads to extremely high performance requirements for the cloud side system, this solution can adopt a cloud-edge collaborative architecture, thus making it easier to achieve system expansion and greatly reducing the limitations on system scalability.

[0110] Based on an object search method provided from the perspective of edge nodes, embodiments of this disclosure also provide an object search method from the perspective of a cloud-side processing system. For example... Figure 2As shown, from the perspective of a cloud-side processing system, an object search method provided in this disclosure embodiment may include the following steps:

[0111] S201, Obtain the target image containing the object to be searched sent by the requester, and send a specified command to the edge node;

[0112] The number of edge nodes can be one or more.

[0113] It is understandable that if there are multiple edge nodes, issuing specific commands to the edge nodes may include:

[0114] According to the search assistance information corresponding to the target image, an edge node to be utilized is selected from a plurality of edge nodes; wherein, the search assistance information includes information characterizing the search area corresponding to the target image;

[0115] Send the specified command to the selected edge node.

[0116] The search assistance information includes information representing the search area corresponding to the target image. The purpose of this search assistance information is to instruct the cloud-side processing system to select edge nodes to be utilized. It is understood that the search assistance information can be specified by the user when uploading the target image. For example, when uploading the target image, the user can specify the desired search area. After the search area is determined, the mapping relationship between the area and the monitoring device, and the mapping relationship between the monitoring device and the edge node, can be used to determine the edge nodes to be utilized. Alternatively, the search assistance information can be a specific marker or feature in the target image. For example, if the target image contains a distinctive building, this information can represent the search assistance information of the target image. Furthermore, the search assistance information can be used to determine the search area or the corresponding monitoring device, thereby using the aforementioned mapping relationship to determine the edge nodes to be utilized.

[0117] S202, Obtain a specified image sent by the edge node; wherein, the specified image is generated by the edge node in response to a specified command issued by the cloud-side processing system, and uploaded to the cloud-side processing system according to a specified generation method, the specified generation method including obtaining video data collected by the target monitoring device; determining candidate objects existing in the video data; and extracting the object region of the candidate object from the image containing the candidate object in the video data to obtain the specified image;

[0118] S203, perform matching analysis on the objects in the target image and the objects in the specified image to obtain the analysis results;

[0119] For cloud-side processing systems, after acquiring the target image and the specified image, matching analysis can be performed on the objects in the target image and the objects in the specified image to obtain analysis results. It should be noted that the matching analysis of the objects in the target image and the candidate objects in the specified image involves analyzing the object features of the objects in the target image and the candidate objects in the specified image. Therefore, object feature extraction is required for both the target image and the specified image. For example, this object feature extraction can be accomplished using ResNet101 (a 101-layer residual network). Residual networks are characterized by ease of optimization and the ability to improve accuracy by increasing their depth. The object features of the candidate objects in the specified image are used as the first object features, and the object features of the searched object in the target image are used as the second object features. For example, the cloud-side processing system uses ResNet101 to extract object features such as the hairstyle and clothing of pedestrian A in the specified image as the first object features; the cloud-side processing system uses ResNet101 to extract the features of the objects in the target image as the second object features.

[0120] There are various specific implementation methods for matching and analyzing objects in the target image and objects in the specified image. For example, in one implementation method, matching and analyzing objects in the target image and objects in the specified image to obtain analysis results may include step D1:

[0121] Step D1: Using a pedestrian re-identification algorithm, analyze whether the object in the specified image matches the object in the target image to obtain the analysis result.

[0122] Pedestrian re-identification algorithms utilize computer vision technology to determine whether pedestrian images appearing at different times and under different surveillance conditions belong to the same person. Due to factors such as camera quality, imaging light, imaging angle, and imaging distance, the features of people obtained from surveillance videos are often unclear. Therefore, feature extraction is performed on target pedestrian images based on characteristics such as clothing, posture, body proportions, and facial features. This can be combined with pedestrian detection and tracking technologies to overcome the current visual limitations of surveillance cameras and can be widely applied in monitoring, security, and other fields. Essentially, it can be viewed as an image matching task; given an image of a pedestrian to be matched, other photos of that person can be searched from a database.

[0123] It should be noted that the pedestrian re-identification algorithm can perform matching analysis between the aforementioned first object features and the aforementioned second object features. For example, the pedestrian re-identification algorithm can perform matching analysis between the clothing, head shape, and other object features of candidate objects in a specified image and the clothing, head shape, and other object features of the search object in the target image to determine whether the pedestrian images appearing in the specified image and the target image belong to the same object.

[0124] For example, in another implementation, Mahalanobis distance can be used to evaluate the similarity between objects in the target image and candidate objects in a specified image, thereby obtaining the analysis results. Mahalanobis distance is an efficient method for calculating the similarity between two unknown sample sets.

[0125] S204, if the analysis result shows a match, feedback is sent to the edge node indicating that the object in the specified image has been successfully matched, so that the edge node, upon receiving the feedback information, sends the search result corresponding to the target image to the requester based on the image containing the candidate object in the video data.

[0126] The implementation methods of the relevant steps S201, S202 and S204 can be found in the above-described embodiment from the perspective of edge nodes, and will not be repeated here.

[0127] In this embodiment, the object search method is implemented through cloud-edge collaboration. This eliminates the need to upload video data from the monitoring device to the cloud-side processing system; instead, the data is uploaded to edge nodes. These edge nodes then perform video analysis and upload a small, specified image to the cloud-side processing system. The cloud-side processing system then matches and analyzes the specified image against the target image. Simultaneously, the search results are directly fed back to the requesting party through the edge nodes. Therefore, compared to existing technologies, this cloud-edge collaboration approach to object search reduces network bandwidth consumption.

[0128] Based on the above object search method, this disclosure also provides an object search system. The object search system may include: a cloud-side processing system and edge nodes;

[0129] The cloud-side processing system is used to send a specified command to the edge node after obtaining the target image containing the object to be searched sent by the requester.

[0130] The edge node is configured to respond to the specified command by acquiring video data collected by the target monitoring device; identifying candidate objects in the video data; extracting the object region of the candidate object from the image containing the candidate object in the video data to obtain a specified image, and sending the specified image to the cloud-side processing system; the cloud-side processing system is further configured to acquire the specified image sent by the edge node; perform matching analysis on the object in the target image and the object in the specified image to obtain the analysis result; if the analysis result shows a match, it sends back an indication message to the edge node indicating that the object in the specified image has been successfully matched;

[0131] The edge node is also configured to, upon receiving the indication information, provide the requester with the search results corresponding to the target image based on the image of the candidate object contained in the video data.

[0132] In this embodiment, the object search system is implemented through cloud-edge collaboration. This eliminates the need to upload video data from the monitoring device to the cloud-side processing system; instead, the data is uploaded to edge nodes. These edge nodes then perform video analysis and upload a small, specified image to the cloud-side processing system. The cloud-side processing system then matches and analyzes the specified image against the target image. Simultaneously, the search results are directly fed back to the requesting party through the edge nodes. Therefore, compared to existing technologies, this cloud-edge collaborative object search method reduces network bandwidth consumption.

[0133] In addition, by extracting the object region of the candidate object from the image containing the candidate object in the video data, the network bandwidth consumption is reduced compared to directly uploading the video data, while also protecting the privacy of the monitoring site to a certain extent. Furthermore, compared to the centralized deployment of object search inference services on the cloud side, which leads to extremely high performance requirements for the cloud side system, this solution can adopt a cloud-edge collaborative architecture, thus making it easier to achieve system expansion and greatly reducing the limitations on system scalability.

[0134] The following description, in conjunction with the accompanying drawings, introduces an object search system provided by an embodiment of this disclosure.

[0135] As shown in Figure 3(a), an object search system may include:

[0136] The cloud-side processing system 300 is used to send a specified command to the edge node after obtaining the target image containing the object to be searched sent by the requester.

[0137] The edge node 330 is used to respond to the specified command to acquire video data collected by the target monitoring device; determine candidate objects existing in the video data; extract the object region of the candidate object from the image containing the candidate object in the video data to obtain a specified image, and send the specified image to the cloud-side processing system.

[0138] The cloud-side processing system 300 is also used to acquire the specified image sent by the edge node; perform matching analysis on the objects in the target image and the objects in the specified image to obtain the analysis result; if the analysis result shows that they match, it feeds back to the edge node an indication that the objects in the specified image have been successfully matched;

[0139] The edge node 330 is also configured to, upon receiving the indication information, provide the requester with the search results corresponding to the target image based on the image of the candidate object contained in the video data.

[0140] The specific implementation methods of the functions implemented by the cloud-side processing system 300 and the edge node 330 can be found in the relevant content of the above method embodiments, and will not be repeated here.

[0141] It should be noted that the functions implemented by the cloud-side processing system 300 can be integrated into a single node or implemented through multiple nodes. For example, the functions implemented by the cloud-side processing system 300 can be achieved through the cooperation of two types of nodes: cloud-side worker nodes and cloud-side control centers. Specifically, for the case where the functions implemented by the cloud-side processing system 300 are achieved through the cooperation of cloud-side worker nodes and cloud-side control centers, the structural diagram of the object search system can be seen in Figure 3(b). The object search system includes: a cloud-side worker node 310, a cloud-side control center 320, and an edge node 330. The cloud-side worker node 310 is communicatively connected to both the cloud-side control center 320 and the edge node 330, and the cloud-side control center 320 is communicatively connected to the edge node 330.

[0142] It should be noted that the cloud-side worker node 310 can be composed of high-performance cloud nodes and deployed with pedestrian re-identification related AI (Artificial Intelligence) models or other models used for object matching analysis; for example, the cloud-side worker node 310 can be a high-performance computer deployed with AI models. Pedestrian re-identification is a technique that uses computer vision technology to determine whether pedestrian images appearing at different times and under different monitoring conditions belong to the same person. In other words, the cloud-side worker node 310 is specifically used to implement the process of matching and analyzing objects in the target image and objects in the specified image.

[0143] The cloud-side control center 320 can be a cloud node or node cluster that interfaces with user control commands and issues commands to the edge node 330 and the cloud-side worker node 310. In other words, the cloud-side control center 320 is used to receive target images sent by the requester, issue specified commands to the edge node 330, and receive target images reported by the edge node 330; furthermore, it selects the cloud-side worker node 310 and sends the target image and specified image to the cloud-side worker node 310 so that the cloud-side worker node 310 can analyze and process the target image and specified image to obtain analysis results.

[0144] Additionally, it should be noted that the cloud-side worker node 310, edge node 330, and cloud-side control center 320 in this embodiment can implement corresponding business functions through containers. Containers can package applications, which can simplify the process of building, deploying, and running applications.

[0145] Optionally, given that the cloud-side processing system includes cloud-side worker nodes 310 and a cloud-side control center 320, and the number of edge nodes 330 is multiple, the structural diagram of the object search system can be as follows: Figure 4 As shown.

[0146] Furthermore, it's understandable that when there are multiple cloud-side worker nodes 310, if the cloud-side control center 320 is a cloud node, it can deploy a load balancing module, thus enabling load balancing. Alternatively, if the cloud-side control center 320 is a node cluster, a load balancing module can be deployed on one of its nodes, allowing load balancing to be achieved through that node – this is also reasonable. It's understandable that the cloud-side control center 320, following load balancing principles, selects a cloud-side worker node 310 with sufficient resources to analyze the target image from among the multiple cloud-side worker nodes 310. This improves the efficiency of object search.

[0147] In another implementation, when there are multiple cloud-side worker nodes, the cloud-side control center 320 can send load balancing requests to other nodes, enabling them to select a cloud-side worker node 310 to be used from among the multiple cloud-side worker nodes 310 according to load balancing principles. In this case, the other nodes communicating with the cloud-side control center 320 have load balancing modules deployed, and the cloud-side control center 320 can perform load balancing processing through these other nodes.

[0148] In this embodiment, the object search system is implemented through cloud-edge collaboration. This eliminates the need to upload video data from the monitoring device to the cloud-side processing system; instead, the data is uploaded to edge nodes. These edge nodes then perform video analysis and upload a small, specified image to the cloud-side processing system. The cloud-side processing system then matches and analyzes the specified image against the target image. Simultaneously, the search results are directly fed back to the requesting party through the edge nodes. Therefore, compared to existing technologies, this cloud-edge collaborative object search method reduces network bandwidth consumption.

[0149] In addition, by extracting the object region of the candidate object from the image containing the candidate object in the video data, the network bandwidth consumption is reduced compared to directly uploading the video data, while also protecting the privacy of the monitoring site to a certain extent. Furthermore, compared to the centralized deployment of object search inference services on the cloud side, which leads to extremely high performance requirements for the cloud side system, this solution can adopt a cloud-edge collaborative architecture, thus making it easier to achieve system expansion and greatly reducing the limitations on system scalability.

[0150] To further understand the working principle of the object search system, the following section describes the object search process from the perspective of user interaction with the system. It's important to understand that the cloud-side processing system can be divided into cloud-side worker nodes and a cloud-side control center; this clarifies the interaction process. For example... Figure 5 As shown, from the perspective of user interaction with the object search system, the object search process may include the following steps:

[0151] S501, User uploads target image;

[0152] In this context, the device used by the user is the requester, such as a streaming media server, client, etc.; the target image includes the object to be searched.

[0153] For example, a user uses a streaming media server to upload a target image, including the object to be searched, to the cloud-side control center;

[0154] S502, the cloud-side worker node extracts object features from the target image;

[0155] Specifically, after receiving the target image uploaded by the user, the cloud-side control center sends the target image to the cloud-side worker node. The cloud-side worker node extracts object features from the search object in the target image; these object features are easily visible characteristics in the image, such as the search object's clothing and hairstyle.

[0156] For example, a cloud-side worker node extracts object features from a target image.

[0157] S503, users assign source video addresses to edge nodes;

[0158] Among them, the video source address is assigned to the address that is closer to the edge node;

[0159] S504, edge nodes connect to the source video stream and decode the source video;

[0160] For example, after receiving the user's video source address, the edge node connects with the source video stream and decodes the source video;

[0161] S505, edge node recognition and tagging of candidate objects in videos;

[0162] For example, after obtaining the source video, the edge nodes identify and mark the locations of candidate objects;

[0163] S506, edge nodes crop candidate objects based on the minimum bounding box;

[0164] For example, the edge node crops the positions of the marked candidate objects according to the minimum bounding box to obtain the specified image; the edge node sends the specified image to the cloud-side control center;

[0165] S507, the cloud-side worker node received a re-identification service request;

[0166] For example, the cloud-side worker node receives a re-identification service request sent by the cloud-side control center;

[0167] S508, the cloud-side worker node determines whether the candidate object in the specified image matches the search object in the target image;

[0168] For example, the cloud-side worker node extracts the object features of candidate objects in a specified image, such as clothing and hairstyle, and matches the object features of the candidate objects in the specified image with the object features of the search object in the target image; if the matching result is successful, S509 is executed; if the matching result is unsuccessful, the process loops back to S505.

[0169] S509, the edge node sends the search results corresponding to the target image back to the requester;

[0170] For example, if the matching result is successful, the edge node feeds back to the user's streaming media device a trajectory video containing candidate objects that successfully matched the target image.

[0171] In summary, this solution implements object search through cloud-edge collaboration, which can reduce network bandwidth consumption.

[0172] In addition, by extracting the object region of the candidate object from the image containing the candidate object in the video data, the network bandwidth consumption is reduced compared to directly uploading the video data, while also protecting the privacy of the monitoring site to a certain extent. Furthermore, compared to the centralized deployment of object search inference services on the cloud side, which leads to extremely high performance requirements for the cloud side system, this solution can adopt a cloud-edge collaborative architecture, thus making it easier to achieve system expansion and greatly reducing the limitations on system scalability.

[0173] To better understand object search systems, the following will combine... Figure 6 and Figure 7This section introduces the working principle of the object search system. To facilitate this explanation, the cloud-side processing system is divided into two types of nodes: cloud-side worker nodes and cloud-side control center nodes. Figure 6 As shown, the object search system is divided into a cloud-side part and an edge-side part. The cloud-side part includes cloud-side worker nodes and a cloud-side control center, while the edge-side part includes edge nodes. The cloud-side worker nodes embed a pedestrian re-identification model, which works based on a pedestrian re-identification algorithm and can determine whether pedestrian images appearing in different time periods and under different monitoring conditions belong to the same person.

[0174] The cloud-side control center contains a resource scheduler and a model image repository. It can also be used for state storage, resource monitoring, and service proxying. The resource scheduler can allocate video source addresses to edge nodes based on configuration files. Since the edge nodes have built-in pedestrian detection modules, they possess a certain level of inference capability. Upon receiving a user's instruction to initiate the recognition and labeling of candidate objects, the cloud-side control center activates the pedestrian detection model of the edge nodes.

[0175] It should be noted that the model image repository serves both cloud-side worker nodes and edge nodes. When the system starts up or when a new worker node needs to be added to the cloud-side worker node, the cloud-side worker node pulls the corresponding business image from the image repository to run the corresponding business model and automatically initializes it. Also, if no model is deployed on the edge node (corresponding to the specified business module in the above embodiment), the image file can be pulled from the model image repository.

[0176] It should be noted that the resource monitoring function of the cloud-side control center can monitor the working status of cloud-side worker nodes in real time. For edge nodes, when an unexpected error occurs during business execution, the video source address being processed by the corresponding node will be reassigned to other nodes, and detailed error information will be reported and recorded in the system log in real time.

[0177] It should be noted that the cloud-side control center's state storage function can store the working status of cloud-side worker nodes and edge nodes in real time. When errors occur in the execution of business processes by cloud-side worker nodes or edge nodes, the cloud-side control center can store the current workload to prevent the current workload from being lost.

[0178] When assigning tasks to cloud-side worker nodes from the cloud-side control center, it is often necessary to use the help of a load balancing module, which can distribute tasks to cloud-side worker nodes with sufficient resources.

[0179] The load balancing module can balance and distribute the load and tasks across multiple operating units. For example, it can allocate tasks to network servers, core enterprise application servers, and other major task servers to collaboratively complete the tasks. If the cloud-side control center is a cloud node, it can deploy a load balancing module, thus enabling load balancing. Alternatively, if the cloud-side control center is a node cluster, a load balancing module can be deployed on a specific node, allowing the control center to achieve load balancing through that node – this is also reasonable.

[0180] Understandably, the cloud-side control center selects cloud-side worker nodes with sufficient resources to analyze the target image from multiple cloud-side worker nodes, based on load balancing principles. This improves the efficiency of object search.

[0181] In another implementation, the cloud-side control center can send load balancing requests to other nodes, enabling these nodes to select a suitable cloud-side worker node from among multiple cloud-side worker nodes according to load balancing principles. In this case, the other nodes communicating with the cloud-side control center have load balancing modules deployed, allowing the cloud-side control center to perform load balancing processing through these other nodes.

[0182] The edge node includes a video docking / preprocessing module and a pedestrian detection model. The edge node needs to connect to the camera on the edge to obtain the source video from the source address. After docking with the source video stream, it also needs to preprocess the video, such as decoding the video and cropping candidate objects from specified images corresponding to the source video using minimum bounding boxes. The edge node can also connect to the user's streaming media server.

[0183] In addition, such as Figure 7 As shown, the user uploads a target image, and the edge node takes a screenshot of the video data from the monitoring device to obtain a screenshot of the person in the monitoring video (corresponding to the specified image in the above embodiment); the cloud-side control center selects a cloud-side worker node to be used through load balancing; the cloud-side worker node to be used can obtain the target image, as well as the screenshot of the person in the monitoring video and the person ID from the Re-ID request; the cloud-side worker node to be used can use the ResNet101 feature extraction network in the Re-ID model to extract object features from the target image and the person screenshot, and then use the extracted object features to perform similarity discrimination. Based on the similarity discrimination result, the result is returned to the edge node, where the matching result indicates whether the match is successful. If the match is successful, the target person ID of the successfully matched person can be fed back.

[0184] The cloud-side control center can achieve load balancing via a REST API to distribute the target images and personnel screenshots to Re-ID models on cloud-side worker nodes with sufficient resources. The Re-ID model includes a ResNet101 feature extraction network and a similarity discrimination module. The ResNet101 feature extraction network extracts features from the target images and personnel screenshots. The similarity discrimination module performs similarity matching between the features of the target images and personnel screenshots, returning the result. If a match is successful, it returns the matching result and candidate object features. The REST API can be an API from other nodes communicating with the cloud-side control center. Calling this REST API enables load balancing analysis, i.e., selecting the cloud-side worker nodes to utilize. Compared to existing technologies, this system delegates tasks such as connecting to the source video stream, acquiring the source video, and performing pedestrian detection on candidate objects in the video data of monitoring equipment to edge nodes deployed on the edge. In existing technologies, these tasks are all performed by cloud-side worker nodes. This means that cloud-side worker nodes need to transmit the acquired source video over the public internet, resulting in high latency and privacy risks. Understandably, deploying the entire system on the cloud in existing technologies reduces system scalability. This system, however, uses a cloud-edge collaborative deployment approach, which provides better elastic scalability. Pedestrian detection is handled by the edge nodes closer to the source video acquisition nodes, while pedestrian re-identification, requiring significant computing power, is handled by cloud-side worker nodes, thus improving the overall system's analysis efficiency. It should also be noted that this cloud-edge collaborative deployment approach reduces network bandwidth consumption compared to existing technologies.

[0185] To better understand the solution, the working principle of a pedestrian detection module provided in this disclosure embodiment, namely the working principle of the specified business module in the aforementioned edge node, will be introduced below. For example... Figure 8 As shown, the edge nodes input a video containing candidate objects to the pedestrian detection module. The pedestrian detection module can divide the video into multiple frames, and perform pedestrian detection, feature extraction, and pedestrian feature association on each frame. When associating pedestrian features, the tracking result is obtained by associating with the pedestrian features of the previous frame or the previous k frames.

[0186] For example, after inputting a video including pedestrian C into the pedestrian detection module, the pedestrian detection module can divide the video into video frames Pi, Pi+1, Pi+2, ..., Pi+k, and perform pedestrian detection, feature extraction, and pedestrian feature association on each frame. During feature extraction, the object features of pedestrian C are extracted. During pedestrian feature association, by associating with the object features of the previous k frames, the continuous positions of pedestrian C in the video are obtained, thus obtaining the tracking result, that is, the trajectory of pedestrian C in the video.

[0187] This embodiment of the disclosure can perform feature association on pedestrians in a video frame by frame to obtain the continuous positions of pedestrians in the video, thereby obtaining the trajectory of pedestrians in the video.

[0188] Based on the above-described method embodiments provided from edge nodes, this disclosure provides an object search device applied to edge nodes, such as... Figure 9 As shown, the device includes:

[0189] The first acquisition module 910 is used to acquire video data collected by the target monitoring device in response to a specified command issued by the cloud-side processing system; wherein, the specified command is issued by the cloud-side processing system to the edge node after acquiring a target image containing the object to be searched sent by the requester;

[0190] The determination module 920 is used to determine the candidate objects present in the video data;

[0191] The cropping module 930 is used to crop the object region of the candidate object from the image containing the candidate object in the video data to obtain a specified image;

[0192] The sending module 940 is used to send the specified image to the cloud-side processing system so that the cloud-side processing system can perform matching analysis on the objects in the specified image and the objects in the target image to obtain the analysis result; if the analysis result shows that they match, it feeds back to the edge node an indication that the objects in the specified image have been successfully matched.

[0193] The first feedback module 950 is used to, upon receiving the instruction information, provide the requesting party with the search results corresponding to the target image based on the image of the candidate object contained in the video data.

[0194] Optionally, the module is determined, including:

[0195] A determination submodule is used to determine candidate objects existing in the video data using a specified business module; wherein, the specified business module is a module in the form of a container generated by running a target image used to determine candidate objects.

[0196] Optionally, the device is further configured to, before determining the candidate object in the video data using the specified service module, if it is detected that the specified service module does not exist, obtain the target image from the image library of the cloud-side processing system;

[0197] Run the target image to generate a container-shaped module, which will serve as the specified business module.

[0198] Optionally, the first analysis module includes:

[0199] The calling module is used to call the specified business module so that the specified business module performs object location detection on the video data, uses the object location detection results to generate the trajectory of the object in the video data, and determines the object indicated by the trajectory as a candidate object in the video data.

[0200] Based on the above-described method embodiments from the perspective of cloud-side processing systems, this disclosure also provides an object search device applied to cloud-side processing systems, such as... Figure 10 As shown, the device includes:

[0201] The second acquisition module 1010 is used to acquire the target image containing the object to be searched sent by the requester and send a specified command to the edge node;

[0202] The third acquisition module 1020 is used to acquire a specified image sent by the edge node; wherein, the specified image is generated by the edge node in response to a specified command issued by the cloud-side processing system, and uploaded to the cloud-side processing system according to a specified generation method, the specified generation method including acquiring video data collected by the target monitoring device; determining candidate objects existing in the video data; and extracting the object region of the candidate object from the image containing the candidate object in the video data to obtain the specified image;

[0203] Analysis module 1030 is used to perform matching analysis on objects in the target image and objects in the specified image to obtain analysis results;

[0204] The second feedback module 1040 is used to provide feedback to the edge node if the analysis result shows a match, indicating that the object in the specified image has been successfully matched, so that the edge node, upon receiving the indication information, provides feedback to the requester with the search results corresponding to the target image based on the image containing the candidate object in the video data.

[0205] Optionally, the analysis module includes:

[0206] The analysis submodule is used to analyze whether the objects in the specified image match the objects in the target image using a pedestrian re-identification algorithm, and obtain the analysis results.

[0207] Optionally, the number of edge nodes can be multiple;

[0208] The process of issuing specified commands to edge nodes includes:

[0209] According to the search assistance information corresponding to the target image, an edge node to be utilized is selected from a plurality of edge nodes; wherein, the search assistance information includes information characterizing the search area corresponding to the target image;

[0210] Send the specified command to the selected edge node.

[0211] This disclosure also provides an electronic device, such as... Figure 11 As shown, it includes a processor 1101, a communication interface 1102, a memory 1103, and a communication bus 1104. The processor 1101, communication interface 1102, and memory 1103 communicate with each other via the communication bus 1104.

[0212] Memory 1103 is used to store computer programs;

[0213] The processor 1101 is used to implement the above-mentioned object search method when executing the program stored in the memory 1103.

[0214] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0215] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0216] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0217] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0218] In another embodiment provided in this disclosure, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described object search methods.

[0219] In yet another embodiment provided in this disclosure, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the object search methods described above.

[0220] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this disclosure are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)).

[0221] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0222] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0223] The above description is merely a preferred embodiment of this disclosure and is not intended to limit the scope of protection of this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure are included within the scope of protection of this disclosure.

Claims

1. A method of searching for an object, characterized by, The method applied to an edge node comprises: In response to a specified command issued by a cloud-side processing system, obtaining video data collected by a target monitoring device; wherein the specified command is issued by the cloud-side processing system to the edge node after the cloud-side processing system obtains a target picture containing a to-be-searched object sent by a requester; Determining a candidate object existing in the video data; From a picture of the video data containing the candidate object, an object region of the candidate object is intercepted to obtain a specified picture; Sending the specified picture to the cloud-side processing system, so that the cloud-side processing system performs matching analysis on the object in the specified picture and the object in the target picture to obtain an analysis result; if the analysis result indicates a match, the edge node is fed back with indication information representing that the object in the specified picture matches successfully; After receiving the indication information, based on the picture of the video data containing the candidate object, the requester is fed back with a search result corresponding to the target picture.

2. The method of claim 1, wherein, The method further comprises: If it is detected that the specified business module does not exist, the target image is obtained from an image library of the cloud-side processing system; 3. The method of claim 2, wherein, The target image is run to generate a container-shaped module as the specified business module. The method further comprises: The specified business module is called to enable the specified business module to perform object position detection on the video data, utilize an object position detection result to generate a track of the object in the video data, and determine the object indicated by the track as the candidate object existing in the video data.

4. The method of claim 2, wherein, The method applied to a cloud-side processing system comprises: Obtaining a target picture containing a to-be-searched object sent by a requester, and issuing a specified command to an edge node; 5. An object search method characterized by, Obtaining a specified picture sent by the edge node; wherein the specified picture is generated by the edge node in a specified generation manner and uploaded to the cloud-side processing system in response to the specified command issued by the cloud-side processing system, and the specified generation manner comprises obtaining video data collected by a target monitoring device; determining a candidate object existing in the video data; and from a picture of the video data containing the candidate object, an object region of the candidate object is intercepted to obtain a specified picture; Performing matching analysis on the object in the target picture and the object in the specified picture to obtain an analysis result; and If the analysis result indicates a match, the edge node is fed back with indication information representing that the object in the specified picture matches successfully. ​ If the analysis result indicates a match, the edge node is fed back indication information indicating that the object in the specified picture matches successfully, so that the edge node feeds back the search result corresponding to the target picture to the requester based on the picture containing the candidate object in the video data after receiving the indication information.

6. The method of claim 5, wherein, The analysis result includes: The pedestrian re-identification algorithm is used to analyze whether the object in the specified picture matches the object in the target picture, and an analysis result is obtained.

7. The method of claim 5, wherein, The number of edge nodes is multiple; The specified command includes: According to the search auxiliary information corresponding to the target picture, an edge node to be used is selected from multiple edge nodes; wherein the search auxiliary information includes information indicating a search area corresponding to the target picture; The selected edge node is fed back the specified command.

8. An object search system characterized by comprising: The system includes a cloud-side processing system and edge nodes; The cloud-side processing system is configured to, after obtaining a target picture containing a to-be-searched object sent by a requester, feed a specified command to the edge nodes; The edge node is configured to, in response to the specified command, obtain video data collected by a target monitoring device; The edge node is configured to determine a candidate object existing in the video data, and obtain a specified picture by extracting an object region of the candidate object from a picture containing the candidate object in the video data, and send the specified picture to the cloud-side processing system; The cloud-side processing system is further configured to obtain the specified picture sent by the edge node, and perform matching analysis on the object in the target picture and the object in the specified picture to obtain an analysis result; If the analysis result indicates a match, the edge node is fed back indication information indicating that the object in the specified picture matches successfully; The edge node is further configured to, after receiving the indication information, feed back the search result corresponding to the target picture to the requester based on the picture containing the candidate object in the video data.

9. An object search apparatus characterized by comprising: The device applied to the edge node includes: The first obtaining module is configured to, in response to a specified command fed back by a cloud-side processing system, obtain video data collected by a target monitoring device; wherein the specified command is fed back by the cloud-side processing system after obtaining a target picture containing a to-be-searched object sent by a requester; The determining module is configured to determine a candidate object existing in the video data; The cutting module is configured to obtain a specified picture by extracting an object region of the candidate object from a picture containing the candidate object in the video data; The sending module is configured to send the specified picture to the cloud-side processing system, so that the cloud-side processing system performs matching analysis on the object in the specified picture and the object in the target picture to obtain an analysis result; if the analysis result indicates a match, the edge node is fed back indication information indicating that the object in the specified picture matches successfully. The first feedback module is configured to, after receiving the indication information, feed back, to the requester, a search result corresponding to the target picture based on a picture containing the candidate object in the video data.

10. An object search apparatus characterized by comprising: The device is applied to a cloud-side processing system, and the device comprises: The second acquisition module is configured to acquire a target picture containing a to-be-searched object sent by a requester, and send a specified command to an edge node. The third acquisition module is configured to acquire a specified picture sent by the edge node, wherein the specified picture is generated by the edge node according to a specified generation manner and uploaded to the cloud-side processing system in response to the specified command sent by the cloud-side processing system, and the specified generation manner comprises: acquiring video data collected by a target monitoring device; determining a candidate object existing in the video data; and obtaining a specified picture by intercepting an object region of the candidate object from a picture containing the candidate object in the video data. The analysis module is configured to perform matching analysis on objects in the target picture and objects in the specified picture to obtain an analysis result. The second feedback module is configured to, if the analysis result indicates a match, feed back, to the edge node, indication information representing that the objects in the specified picture match successfully, so that the edge node feeds back, to the requester, a search result corresponding to the target picture based on a picture containing the candidate object in the video data after receiving the indication information.

11. An electronic device, comprising: The device comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are in communication with each other through the communication bus. The memory is configured to store a computer program. The processor is configured to execute the program stored in the memory to implement the method steps in any one of claims 1-7.

12. A computer-readable storage medium, characterized in that, The computer program is stored in the computer-readable storage medium, and the computer program is executed by the processor to implement the method steps in any one of claims 1-7.

Images