Image recognition data transmission method, device and system and electronic equipment

Abstract

The embodiment of the invention relates to an image recognition data transmission method, device and system and electronic equipment. The method comprises the following steps: acquiring image data shot by a target camera; performing character behavior recognition on the image data to obtain character behavior data; determining whether the image data and the character behavior data meet uploading conditions or not; and if yes, sending the image data and the character behavior data to a server, so that the server stores the image data and the character behavior data. According to the embodiment of the invention, the image and the identified character behavior data are screened, and the image and character behavior data meeting the conditions are uploaded, so that the data volume uploaded to the server is reduced, the network bandwidth pressure is relieved, and the data transmission efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image recognition data transmission method, device, system and electronic equipment. Background Art

[0002] With the development of image recognition technology, user privacy protection is an issue that needs to be highly valued. In current image recognition scenarios, it is necessary to upload images taken by cameras to cloud servers, or upload recognition results to cloud servers after the terminal device recognizes the images. For example, the home cat-eye system uses a camera installed outside the user's home to shoot and identify visiting people. The captured images and recognition results need to be uploaded to the cloud server. The data transmission process may cause network bandwidth pressure and other problems. Therefore, how to reduce the amount of data uploaded to the server and ensure data transmission efficiency is a problem that needs to be solved. Summary of the invention

[0003] In view of this, in order to solve some or all of the above-mentioned technical problems, the embodiments of the present application provide an image recognition data transmission method, device, system and electronic device.

[0004] In a first aspect, an embodiment of the present application provides an image recognition data transmission method, the method comprising: acquiring image data captured by a target camera; performing character behavior recognition on the image data to obtain character behavior data; determining whether the image data and the character behavior data meet the upload conditions; if so, sending the image data and the character behavior data to a server so that the server stores the image data and the character behavior data.

[0005] In one possible implementation, sending image data and character behavior data to a server includes: encrypting the image data and character behavior data to obtain encrypted image data and encrypted character behavior data, and sending the encrypted image data and encrypted character behavior data to the server.

[0006] In one possible implementation, after performing character behavior recognition on image data to obtain character behavior data, the method further includes: determining whether the image data and / or character behavior data include privacy data; if privacy data is included, using a preset noise addition strategy to add noise to the privacy data to obtain privacy-protected image data and / or character behavior data.

[0007] In a possible implementation, sending the image data and the character behavior data to the server includes: performing block processing on the image data and the character behavior data to obtain at least two block data; and sending the at least two block data to the server.

[0008] In one possible implementation, determining whether image data and character behavior data meet upload conditions includes: determining the behavior category to which the character behavior data belongs, and determining, based on the behavior category, the value category to which the image data and character behavior data belong; if the image data and character behavior data belong to a first value category, determining that the image data and character behavior data meet the upload conditions; if the image data and character behavior data belong to a second value category, and an instruction indicating that the user allows upload is obtained, determining that the image data and character behavior data meet the upload conditions; if the image data and character behavior data belong to a third value category, determining that the image data and character behavior data do not meet the upload conditions.

[0009] In one possible implementation, if the image data and the character behavior data belong to the third value category and after determining that the image data and the character behavior data do not meet the upload conditions, the method further includes: compressing the image data and the character behavior data, and storing the compressed image data and the character behavior data in a preset storage area.

[0010] In one possible implementation, determining whether the image data and the character behavior data meet the upload conditions includes: if it is detected that insufficient local remaining storage space is available, determining that the image data and the character behavior data meet the upload conditions; if a preset high-computing power processing of the image data and the character behavior data is triggered, determining that the image data and the character behavior data meet the upload conditions.

[0011] In a second aspect, an embodiment of the present application provides an image recognition data transmission device, which includes: an acquisition module, used to acquire image data captured by a target camera; an identification module, used to perform character behavior recognition on the image data to obtain character behavior data; a first determination module, used to determine whether the image data and the character behavior data meet the upload conditions; and a sending module, used to send the image data and the character behavior data to a server if they meet the conditions, so that the server stores the image data and the character behavior data.

[0012] In a third aspect, an embodiment of the present application provides an image recognition data transmission system, which includes: a camera, an edge computing device and a server; the camera is connected to the edge computing device, and the edge computing device is connected to the server; the camera is used to shoot the target area to obtain image data; the edge computing device is used to use the image data to execute the above-mentioned image recognition data transmission method, and send the obtained image data and character behavior data to the server; the server is used to store the image data and the character behavior data.

[0013] In one possible implementation, the system also includes a user terminal device, which is connected to a server and / or an edge computing device; the edge computing device is also used to: encrypt image data and character behavior data to obtain encrypted image data and encrypted character behavior data, and send the encrypted image data and encrypted character behavior data to the server; the user terminal device is used to: send data query instructions to the server and / or edge computing device; the server and / or edge computing device is also used to: decrypt the encrypted image data and encrypted character behavior data corresponding to the data query instruction, and send the decrypted image data and character behavior data to the user terminal device.

[0014] In one possible implementation, a trusted execution environment is provided on the edge computing device, and the image recognition data transmission method is executed in the trusted execution environment.

[0015] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising: a memory for storing a computer program; a processor for executing the computer program stored in the memory, and when the computer program is executed, it implements the method of any embodiment of the image recognition data transmission method of the above-mentioned first aspect of the present application.

[0016] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, a method as in any embodiment of the image recognition data transmission method of the first aspect described above is implemented.

[0017] In a sixth aspect, an embodiment of the present application provides a computer program, which includes a computer-readable code. When the computer-readable code runs on a device, the processor in the device implements a method as in any embodiment of the image recognition data transmission method of the first aspect mentioned above.

[0018] The image recognition data transmission method, device, system and electronic device provided in the embodiments of the present application obtain the human behavior data by performing human behavior recognition on the image data taken by the target camera, and when it is determined that the image data and the human behavior data meet the upload conditions, the image data and the human behavior data are sent to the server for storage. The embodiments of the present application realize that after the image is recognized, the image and the recognized human behavior data are screened, and the images and human behavior data that meet the conditions are uploaded, thereby reducing the amount of data uploaded to the server, alleviating the pressure on the network bandwidth, and improving the data transmission efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0021] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0022] Figure 1 A flowchart of an image recognition data transmission method provided in an embodiment of the present application;

[0023] Figure 2 A flowchart of another method for transmitting image recognition data provided by an embodiment of the present application;

[0024] Figure 3 A flowchart of another method for transmitting image recognition data provided by an embodiment of the present application;

[0025] Figure 4 A flowchart of another method for transmitting image recognition data provided by an embodiment of the present application;

[0026] Figure 5 A schematic diagram of the structure of an image recognition data transmission device provided in an embodiment of the present application;

[0027] Figure 6 A structural diagram of an image recognition data transmission system provided in an embodiment of the present application;

[0028] Figure 7 A schematic diagram of the structure of another image recognition data transmission system provided in an embodiment of the present application;

[0029] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. It should be noted that unless otherwise specifically stated, the relative arrangement of the components and steps, the numerical expressions and the numerical values ​​described in these embodiments do not limit the scope of the present application.

[0031] Those skilled in the art will understand that the terms "first" and "second" in the embodiments of the present application are only used to distinguish between different steps, devices, modules and other objects, and do not represent any specific technical meanings, nor do they indicate the logical order between them.

[0032] It should also be understood that in this embodiment, “plurality” may refer to two or more than two, and “at least one” may refer to one, two or more than two.

[0033] It should also be understood that any component, data or structure mentioned in the embodiments of the present application can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0034] In addition, the term "and / or" in this application is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this application generally indicates that the associated objects before and after are in an "or" relationship.

[0035] It should also be understood that the description of the various embodiments in this application focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.

[0036] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present application, its application, or uses.

[0037] Technologies, circuits, and devices known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the above-mentioned technologies, circuits, and devices should be considered as part of the specification.

[0038] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0039] It should be noted that, in the absence of conflict, the embodiments in this application and the features in the embodiments can be combined with each other. To facilitate the understanding of the embodiments of the present application, the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0040] In order to solve the technical problems of large data volume and low security in the prior art image data transmission to the cloud, the present application provides an image recognition data transmission method, which can reduce the amount of data transmitted to the cloud and improve the security of data transmission.

[0041] Figure 1 A flow chart of an image recognition data transmission method provided for an embodiment of the present application. This method can be applied to various types of terminal devices, such as edge computing devices, monitoring devices, smart phones, laptops, desktop computers, and one or more electronic devices as terminal devices. These electronic devices are connected to a server and a camera, obtain image data from the camera, execute this method based on the image data, and finally upload the processed data to the server. In addition, the execution subject of this method can be hardware or software. When the above-mentioned execution subject is hardware, the execution subject can be one or more of the above-mentioned electronic devices. For example, a single electronic device can execute this method, or a plurality of electronic devices can cooperate with each other to execute this method. When the above-mentioned execution subject is software, this method can be implemented as multiple software or software modules, or as a single software or software module. It is not specifically limited here.

[0042] like Figure 1 As shown, the method specifically includes:

[0043] Step 101, obtaining image data captured by a target camera.

[0044] In some embodiments, the target camera may be a camera in various scenarios, such as an outdoor surveillance camera, a camera included in a home electronic cat-eye system, a camera on a vehicle, etc. The target camera may shoot various objects within its shooting range and obtain image data in real time.

[0045] Step 102, performing character behavior recognition on the image data to obtain character behavior data.

[0046] In some embodiments, the image data may include a person image, and the electronic device may use a related person behavior recognition method to recognize the person image. Among them, the person behavior recognition method may include face recognition, posture recognition, action recognition, identity recognition, etc. The obtained person behavior data may represent the person's identity, behavior classification and other attributes. For example, in the electronic cat's eye system scenario, if the person is determined to be a stranger based on the face recognition results and posture recognition results, and the action recognition results indicate that the person has opened the door, the generated person behavior data includes identity information indicating the stranger's identity, action information indicating that the person has performed the door opening action, etc.

[0047] Generally, the above-mentioned methods for various recognition of image data can be implemented using a pre-trained neural network model. That is, a large number of sample images are annotated in advance to obtain the annotated character behavior data corresponding to each sample image, and the annotated character behavior data represents the true identity, action, posture and other attributes of the person in the sample image. During training, the sample image data are input into the initial model in sequence to obtain the corresponding predicted character behavior data, and the loss value representing the error between the predicted character behavior data and the annotated character behavior data is calculated using the loss function. Then, the parameters of the initial model are iteratively updated using the gradient descent method and the back propagation method to minimize the loss value. When the model meets the preset end conditions, the model parameters are fixed to obtain the trained character behavior recognition model.

[0048] Step 103, determining whether the image data and character behavior data meet the upload conditions.

[0049] In some embodiments, the above-mentioned upload conditions can be preset. According to the character behavior data, the image data and character behavior data to be uploaded to the server can be screened. For example, according to the character behavior data, it can be determined whether the image data contains stranger images. If it does, it means that there is a risk of illegal intrusion by strangers, and it is determined that the upload conditions are met. For another example, the user can manually control whether to send the image data and character behavior data to the server. If an instruction sent by the terminal used by the user is received, it is determined that the upload conditions are met.

[0050] Step 104: if the image data and the character behavior data are in compliance, the image data and the character behavior data are sent to the server so that the server stores the image data and the character behavior data.

[0051] In some embodiments, the server may store image data and character behavior data, and various operations may be further performed using the stored image data and character behavior data. For example, when a user needs to view image data and character behavior data, the server may send the image data and character behavior data to a terminal device used by the user. When the server further determines that an abnormal situation has occurred based on the character behavior data, a warning message is sent to the terminal device used by the user.

[0052] The image recognition data transmission method provided in the embodiment of the present application performs character behavior recognition on the image data taken by the target camera to obtain character behavior data, and when it is determined that the image data and the character behavior data meet the upload conditions, the image data and the character behavior data are sent to the server for storage. The embodiment of the present application realizes that after the image is recognized, the image and the recognized character behavior data are screened, and the images and character behavior data that meet the conditions are uploaded, thereby reducing the amount of data uploaded to the server, alleviating the pressure on the network bandwidth, and improving the data transmission efficiency.

[0053] In some optional implementations, step 104 may be performed as follows:

[0054] The image data and the character behavior data are encrypted to obtain the encrypted image data and the encrypted character behavior data, and the encrypted image data and the encrypted character behavior data are sent to the server.

[0055] In some embodiments, a preset encryption algorithm (eg, AES-256 algorithm) may be used to encrypt the image data and the character behavior data to obtain encrypted image data and encrypted character behavior data.

[0056] Optionally, the electronic device may receive encryption algorithm parameters input by the user and adjust the security of the encryption algorithm according to the parameters. For example, the encryption algorithm may be switched between AES-256 and AES-128.

[0057] This embodiment helps to improve the security of data transmission by encrypting image data and character behavior data before uploading.

[0058] In some optional implementations, such as Figure 2 As shown, after step 102, the method further includes:

[0059] Step 1021, determining whether the image data and / or the character behavior data include privacy data.

[0060] Specifically, rules for determining privacy data may be pre-set, and privacy determination may be performed on image data and / or character behavior data according to the rules.

[0061] For example, the privacy judgment rules may include at least one of the following: if the image data includes a facial image of a preset person (such as the head of the household or other set person), it is determined that it includes privacy data, and the privacy data may include pixel data of the facial area, and may also include facial feature data, etc.; if the character behavior data includes data that may expose the user's behavior habits or specific activities (such as entry and exit time, actions taken when stopping in front of the door, etc.), it is determined that it includes privacy data, and the privacy data may include character action feature data, time data, etc.

[0062] Step 1022: If the privacy data is included, the privacy data is subjected to noise adding processing using a preset noise adding strategy to obtain privacy-protected image data and / or character behavior data.

[0063] Specifically, various noise adding strategies related to the art may be used to add noise to the private data. For example, a differential privacy algorithm may be used to add noise to the private data.

[0064] Optionally, the electronic device can receive parameters of the noise addition strategy input by the user, and adjust the strength of the noise added according to the parameters. For example, if a differential privacy algorithm is used, the strength of the added noise can be controlled by adjusting the privacy budget ε. A lower ε value adds more noise and provides higher privacy protection, but may reduce the practicality of the data; a higher ε value adds less noise, and the analysis results are more accurate, but the privacy protection is weaker. In actual scenarios, when the user wants to obtain higher privacy protection in an unfamiliar environment, the privacy budget can be adjusted to a lower value (such as ε = 0.5) by operating the terminal device connected to the electronic device. In scenarios where family members or acquaintances visit frequently, users can choose a higher privacy budget (such as ε = 2).

[0065] The privacy-protected image data and / or character behavior data can be further used to perform subsequent steps 103-104. For example, if the image data does not include privacy data, but the character behavior data includes privacy data, the original image data and the privacy-protected character behavior data can be used to perform subsequent steps.

[0066] This embodiment performs privacy data judgment on image data and character behavior data, and performs noise addition processing when privacy information is contained, thereby fully protecting the privacy of image data and character behavior data and further improving the security of data transmission.

[0067] In some optional implementations, such as Figure 3 As shown, step 104 includes:

[0068] Step 1041, performing block processing on the image data and the character behavior data to obtain at least two block data.

[0069] Specifically, the number of data blocks can be set, and the image data and character behavior data can be divided into blocks according to the number of blocks. The data volume of each block can also be set, and the block data containing the above data volume can be extracted from the image data and character behavior data in sequence.

[0070] Step 1042: Send at least two block data to the server.

[0071] Optionally, if the image data and the character behavior data are encrypted first, the encrypted image data and the character behavior data are processed in blocks, and each block of data is sent to the server.

[0072] This embodiment divides the image data and character behavior data into multiple data blocks, so that even if part of the data is stolen during the data transmission process, the thief cannot restore the complete information, thereby improving the security of data transmission.

[0073] In some optional implementations, such as Figure 4 As shown, step 103 includes:

[0074] Step 1031, value classification is performed on the character behavior data to obtain the value category to which the image data and the character behavior data belong.

[0075] Among them, the above-mentioned value categories represent the contribution of image data and character behavior data to the detection of abnormal behavior of characters. That is, the higher the value of the category, the higher the possibility that the detected character behavior is abnormal behavior, and the lower the value of the category, the lower the possibility that the detected character behavior is abnormal behavior. For example, if abnormal behaviors such as wandering and prying doors appear in the character behavior data, or data of unidentified persons are identified, the image data and character behavior data can be determined to be the first value category (i.e., high value category). If the character behavior data only includes data representing normal behavior of characters (normal couriers delivering goods, neighbors passing by, etc.), the image data and character behavior data can be determined to be the second value category (i.e., medium value category). If there is no data representing the behavior of any character in the character behavior data (for example, the image only contains the background and no characters appear), it means that the image data and character behavior data are the third value category (i.e., low value category).

[0076] The electronic device can use a pre-trained classification model to value classify the task behavior data. The classification model can include a neural network model trained using a machine learning method, and can also include a table representing the correspondence between character behavior data and value categories. For example, a large amount of sample behavior data can be labeled with real value categories in advance, and then the sample behavior data can be classified using the initial classification model to obtain predicted value categories. The predicted value categories and the labeled value categories are compared using a loss function, and the parameters of the initial classification model are adjusted according to the error (loss value) between the two until the loss value converges, thereby obtaining a trained classification model.

[0077] Step 1032: If the image data and the character behavior data belong to the first value category, determine that the image data and the character behavior data meet the upload conditions.

[0078] That is, when abnormal people or abnormal behaviors (such as strangers wandering around or prying open a door) are detected in the photographed area, it is determined that the image data and the person behavior data meet the upload conditions. After the image data and the person behavior data meet the upload conditions, the server can further analyze whether the image data and the person behavior data meet the upload conditions, or directly send a warning message to the user or other relevant personnel.

[0079] Step 1033: If the image data and the character behavior data belong to the second value category and an instruction indicating that the user allows uploading is obtained, it is determined that the image data and the character behavior data meet the uploading conditions.

[0080] That is, when it is detected that no abnormal people or abnormal behaviors appear in the photographed area, the electronic device will not actively upload the image data and the person's behavior data, but will upload the image data and the person's behavior data to the server according to the generated instructions when the user performs the upload operation.

[0081] Step 1034: If the image data and the character behavior data belong to the third value category, it is determined that the image data and the character behavior data do not meet the upload conditions.

[0082] For example, when no person appears in the captured image and no person behavior is detected, the electronic device does not upload the image data and the person behavior data, but only saves the image data and the person behavior data locally.

[0083] This embodiment judges the behavior category of the character behavior data and determines the value category of the behavior category to which the character behavior data belongs. This realizes the value degree of the behavior category to which the character behavior data belongs, and specifically judges whether the behavior category to which the character behavior data belongs meets the upload conditions, thereby automatically uploading high-value data to the server in a timely manner, uploading the second-value data according to the user operation, and not uploading the low-value data. This helps to improve the targeted nature of data uploading and reduce the bandwidth resources consumed by data transmission and the storage resources of the server.

[0084] In some optional implementations, after the above step 1034, the method further includes:

[0085] The image data and the character behavior data are compressed and stored in a preset storage area.

[0086] Specifically, the image data and the character behavior data may be compressed, and only thumbnails and metadata (eg, data including attributes indicating shooting time, indicating that the shooting content is empty, etc.) are retained.

[0087] Optionally, for data belonging to the first value category, the complete data (including high-definition image data) can be permanently stored in a preset storage area. For data belonging to the second value category, the data including low-definition images can be stored in a preset storage area.

[0088] This embodiment can effectively reduce the amount of data stored in the electronic device and improve resource utilization by compressing the image data and character behavior data belonging to the third value category before storing them.

[0089] In some optional implementations, step 103 may further include the following steps:

[0090] If it is detected that the remaining local storage space is insufficient, it is determined that the image data and character behavior data meet the upload conditions.

[0091] If the preset large computing power processing of the image data and the character behavior data is triggered, it is determined that the image data and the character behavior data meet the upload conditions.

[0092] Specifically, a lower limit threshold of the local remaining storage space can be set. If the current remaining storage space is less than or equal to the lower limit threshold, subsequent steps can be performed to upload the image data and character behavior data to the server to avoid data loss.

[0093] The above-mentioned high-computing power processing can be a preset data processing method. The high-computing power processing requires high computing power resources, and the local electronic device cannot execute the high-computing power processing method. Therefore, the subsequent steps can be executed to upload the image data and character behavior data to the server, and the server executes the high-computing power processing method. As an example, when a large amount of historical image data and historical character behavior data needs to be mined and processed, a high-computing power processing instruction can be sent to the electronic device. After receiving the instruction, the electronic device automatically sends the image data and character behavior data obtained at present or in the future to the server after being processed in the subsequent steps.

[0094] This embodiment realizes automatic determination of whether image data and character behavior data meet upload conditions when local storage space is insufficient and large computing power processing is required, thereby avoiding data loss, saving local computing power resources, and avoiding equipment failures caused by processing large amounts of data.

[0095] Figure 5 This is a schematic diagram of the structure of an image recognition data transmission device provided in an embodiment of the present application. Specifically comprising:

[0096] An acquisition module 501 is used to acquire image data captured by a target camera;

[0097] The recognition module 502 is used to perform character behavior recognition on the image data to obtain character behavior data;

[0098] A first determination module 503, used to determine whether the image data and the character behavior data meet the upload conditions;

[0099] The sending module 504 is used to send the image data and the character behavior data to the server if they meet the requirements, so that the server can store the image data and the character behavior data.

[0100] In some optional implementations, the sending module is further used to: encrypt the image data and the character behavior data to obtain the encrypted image data and the encrypted character behavior data, and send the encrypted image data and the encrypted character behavior data to the server.

[0101] In some optional implementations, the device also includes: a second determination module, used to determine whether the image data and / or character behavior data include privacy data; and a first processing module, used to, if privacy data is included, add noise to the privacy data using a preset noise addition strategy to obtain privacy-protected image data and / or character behavior data.

[0102] In some optional implementations, the sending module includes: a processing unit, used to perform block processing on the image data and the character behavior data to obtain at least two block data; and a sending unit, used to send the at least two block data to the server.

[0103] In some optional implementations, the first determination module includes: a first determination unit, used to determine the behavior category to which the character behavior data belongs, and determine the value category to which the image data and the character behavior data belong based on the behavior category; a second determination unit, used to determine that the image data and the character behavior data meet the upload conditions if the image data and the character behavior data belong to the first value category; a third determination unit, used to determine that the image data and the character behavior data meet the upload conditions if the image data and the character behavior data belong to the second value category and an instruction indicating that the user allows the upload is obtained; and a fourth determination unit, used to determine that the image data and the character behavior data do not meet the upload conditions if the image data and the character behavior data belong to the third value category.

[0104] In some optional implementations, the device further includes: a compression module, configured to compress the image data and the character behavior data, and store the compressed image data and the character behavior data in a preset storage area.

[0105] In some optional implementations, the first determination module includes: a fifth determination unit, used to determine that the image data and the character behavior data meet the upload conditions if it is detected that insufficient local remaining storage space is detected; a sixth determination unit, used to determine that the image data and the character behavior data meet the upload conditions if a preset high-computing power processing of the image data and the character behavior data is triggered.

[0106] The image recognition data transmission device provided in this embodiment can be as follows Figure 5 The image recognition data transmission device shown in can execute all the steps of the above image recognition data transmission methods, thereby realizing the technical effects of the above image recognition data transmission methods. For details, please refer to the above related description. For the sake of brevity, it will not be repeated here.

[0107] Figure 6 A schematic diagram of the structure of an image recognition data transmission system provided in an embodiment of the present application. The system specifically includes: a camera 601, an edge computing device 602 and a server 603; the camera 601 is connected to the edge computing device 602, and the edge computing device 602 is connected to the server 603. The connection between the devices can be wired or wireless.

[0108] The camera 601 is used to shoot the target area and obtain image data. The target area can be an area in various scenes. For example, when the system is applied to the electronic cat-eye monitoring scene, the target area can be the area in front of the user's door.

[0109] The edge computing device 602 is used to utilize image data to execute the image recognition data transmission method described in the above embodiments, and send the obtained image data and character behavior data to the server 603.

[0110] The edge computing device 602 is a device close to the data source, that is, it is set at a location that is no more than a preset distance from the target area. For example, in the electronic cat-eye monitoring scenario, the edge computing device 602 can be set in the building where the user is located, or in the computer room of the community where the user is located, etc.

[0111] The server 603 is used to store image data and character behavior data. After storing the image data and character behavior data, the server 603 can further perform various operations according to requirements. For example, it can receive a data query request issued by a user and feed back the image data and character behavior data queried by the user to the user terminal. For another example, according to the data processing instruction issued by the user, historical data mining and other operations can be performed on the stored image data and character behavior data.

[0112] The image recognition data transmission system provided in the embodiment of the present application realizes uploading of collected image data after recognition by executing the above-mentioned image recognition data transmission method on the edge computing device, thereby reducing the data processing volume of the server and improving the efficiency of image data processing.

[0113] In some optional implementations, such as Figure 7 As shown, the system also includes a user terminal device 604 , which is connected to a server 603 and / or an edge computing device 602 .

[0114] The edge computing device 602 is also used to: encrypt the image data and the character behavior data to obtain the encrypted image data and the encrypted character behavior data, and send the encrypted image data and the encrypted character behavior data to the server.

[0115] The user terminal device 604 is used to send data query instructions to the server 603 and / or the edge computing device 602.

[0116] The user terminal device 604 may include various devices such as smart phones and personal computers. The user may establish a connection with the server 603 and / or the edge computing device 602 by operating an application or a web page, and send a data query instruction to the server 603 and / or the edge computing device 602. The data query instruction may include time information, data category information, etc.

[0117] The server 603 and / or the edge computing device 602 are also used to: decrypt the encrypted image data and encrypted character behavior data corresponding to the data query instruction, and send the decrypted image data and character behavior data to the user terminal device.

[0118] Among them, the decryption method corresponds to the above encryption method. According to the data query instruction, the image data and user behavior data required by the user can be obtained, and the image data and user behavior data can be fed back to the user terminal device 604.

[0119] This embodiment encrypts the image data and character behavior data before uploading, and sets a data query function on the server and / or edge computing device. In the process of uploading data to the server, the security of data transmission is improved. When the user uses the terminal device to query the required image data and user behavior data, the data can be decrypted and then fed back to the user terminal device, thereby improving the security of data query.

[0120] In some optional implementations, a trusted execution environment (TEE) is provided on the edge computing device 602, and the above-mentioned image recognition data transmission method is executed in the trusted execution environment.

[0121] A trusted execution environment is a secure area built by software and hardware on a computing platform that ensures that the code and data loaded in the secure area are protected in terms of confidentiality and integrity. Its goal is to ensure that a task is executed as expected, ensuring the confidentiality and integrity of the initial state, as well as the confidentiality and integrity of the runtime state.

[0122] By setting up a trusted execution environment on the edge computing device, it can be ensured that all local data processing tasks (such as facial recognition, behavioral analysis, etc.) are performed in a protected environment, so that even if the edge computing device is maliciously attacked or has vulnerabilities, sensitive data cannot be stolen or tampered with, thereby improving data security.

[0123] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application, Figure 8 The electronic device 800 shown includes: at least one processor 801, a memory 802, at least one network interface 804 and other user interfaces 803. The various components in the electronic device 800 are coupled together via a bus system 805. It is understood that the bus system 805 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 805 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 805 is not described in detail. Figure 8 Various buses are labeled as bus system 805 .

[0124] The user interface 803 may include a display, a keyboard, or a pointing device (eg, a mouse, a trackball, a touch pad, or a touch screen).

[0125] It can be understood that the memory 802 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The memory 802 described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0126] In some implementations, the memory 802 stores the following elements, executable units or data structures, or a subset thereof, or an extended set thereof: an operating system 8021 and an application program 8022 .

[0127] The operating system 8021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application 8022 includes various application programs, such as a media player (Media Player), a browser (Browser), etc., which are used to implement various application services. The program for implementing the method of the embodiment of the present application can be included in the application 8022.

[0128] In this embodiment, by calling the program or instruction stored in the memory 802, specifically, the program or instruction stored in the application 8022, the processor 801 is used to execute the method steps provided by each method embodiment, for example, including:

[0129] Acquire image data captured by a target camera; perform character behavior recognition on the image data to obtain character behavior data; determine whether the image data and the character behavior data meet the upload conditions; if so, send the image data and the character behavior data to a server so that the server stores the image data and the character behavior data.

[0130] The method disclosed in the above embodiment of the present application can be applied to the processor 801, or implemented by the processor 801. The processor 801 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 801. The above processor 801 can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software units in the decoding processor can be executed. The software unit can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 802, and the processor 801 reads the information in the memory 802 and completes the steps of the above method in combination with its hardware.

[0131] It is understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPDevice, DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the above functions of the present application, or a combination thereof.

[0132] For software implementation, the technology described above in this article can be implemented by a unit that performs the functions described above in this article. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0133] The electronic device provided in this embodiment may be Figure 8 The electronic device shown in can execute all the steps of the above-mentioned image recognition data transmission methods, thereby realizing the technical effects of the above-mentioned image recognition data transmission methods. For details, please refer to the above related description. For the sake of brevity, it will not be elaborated here.

[0134] The embodiment of the present application also provides a storage medium (computer-readable storage medium). The storage medium here stores one or more programs. The storage medium may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a read-only memory, a flash memory, a hard disk or a solid-state drive; the memory may also include a combination of the above-mentioned types of memory.

[0135] When one or more programs in the storage medium can be executed by one or more processors, the image recognition data transmission method executed on the electronic device side can be implemented.

[0136] The processor is used to execute the program stored in the memory to implement the following steps of the image recognition data transmission method performed on the electronic device side:

[0137] Acquire image data captured by a target camera; perform character behavior recognition on the image data to obtain character behavior data; determine whether the image data and the character behavior data meet the upload conditions; if so, send the image data and the character behavior data to a server so that the server stores the image data and the character behavior data.

[0138] The professionals should further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different circuits to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0139] The steps of the circuits or algorithms described in conjunction with the embodiments disclosed herein may be implemented using hardware, software modules executed by a processor, or a combination of the two. The software modules may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0140] It should be understood that the terms used in the text are only for the purpose of describing specific example embodiments, and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used in the text may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The steps, processes, and operations described in the text are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0141] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.

Claims

1. An image recognition data transmission method, characterized in that: The method comprises: Get the image data captured by the target camera; Performing character behavior recognition on the image data to obtain character behavior data; Determining whether the image data and the character behavior data meet the upload conditions; If it is in compliance, the image data and the character behavior data are sent to a server so that the server stores the image data and the character behavior data.

2. The method according to claim 1, characterized in that The sending the image data and the character behavior data to a server includes: The image data and the character behavior data are encrypted to obtain encrypted image data and encrypted character behavior data, and the encrypted image data and the encrypted character behavior data are sent to a server.

3. The method according to claim 1, characterized in that After performing character behavior recognition on the image data to obtain character behavior data, the method further includes: Determining whether the image data and / or the character behavior data include private data; If private data is included, the private data is subjected to noise adding processing using a preset noise adding strategy to obtain privacy-protected image data and / or character behavior data.

4. The method according to claim 1, characterized in that: The sending the image data and the character behavior data to a server includes: Performing block processing on the image data and the character behavior data to obtain at least two block data; The at least two block data are sent to the server.

5. The method according to claim 1, characterized in that The determining whether the image data and the character behavior data meet the upload conditions includes: Determining the behavior category to which the character behavior data belongs, and determining the value category to which the image data and the character behavior data belong according to the behavior category; If the image data and the character behavior data belong to the first value category, determining that the image data and the character behavior data meet the upload condition; If the image data and the character behavior data belong to the second value category and an instruction indicating that the user allows uploading is obtained, determining that the image data and the character behavior data meet the uploading conditions; If the image data and the character behavior data belong to the third value category, it is determined that the image data and the character behavior data do not meet the upload conditions.

6. The method according to claim 5, characterized in that After determining that the image data and the character behavior data do not meet the upload condition if the image data and the character behavior data belong to the third value category, the method further includes: The image data and the character behavior data are compressed, and the compressed image data and character behavior data are stored in a preset storage area.

7. The method according to any one of claims 1 to 6, characterized in that: The determining whether the image data and the character behavior data meet the upload conditions includes: If it is detected that the remaining local storage space is insufficient, determining that the image data and the character behavior data meet the upload conditions; If the preset large computing power processing of the image data and the character behavior data is triggered, it is determined that the image data and the character behavior data meet the upload conditions.

8. An image recognition data transmission device, characterized in that: The device comprises: An acquisition module is used to acquire image data captured by a target camera; A recognition module, used for performing character behavior recognition on the image data to obtain character behavior data; A first determination module, used to determine whether the image data and the character behavior data meet the upload conditions; The sending module is used to send the image data and the character behavior data to a server if the conditions are met, so that the server can store the image data and the character behavior data.

9. An image recognition data transmission system, characterized in that: The system comprises: a camera, an edge computing device and a server; the camera is connected to the edge computing device, and the edge computing device is connected to the server; The camera is used to shoot the target area to obtain image data; The edge computing device is used to use the image data to execute the image recognition data transmission method according to any one of claims 1 to 7, and send the obtained image data and character behavior data to the server; The server is used to store the image data and the character behavior data.

10. The system according to claim 9, characterized in that The system further includes a user terminal device, wherein the user terminal device is connected to the server and / or the edge computing device; The edge computing device is further used to: encrypt the image data and the character behavior data to obtain encrypted image data and encrypted character behavior data, and send the encrypted image data and the encrypted character behavior data to a server; The user terminal device is used to: send a data query instruction to the server and / or the edge computing device; The server and / or the edge computing device is also used to: decrypt the encrypted image data and encrypted character behavior data corresponding to the data query instruction, and send the decrypted image data and character behavior data to the user terminal device.

11. The system according to claim 9, characterized in that A trusted execution environment is provided on the edge computing device, and the image recognition data transmission method is executed in the trusted execution environment.

12. An electronic device, characterized in that: include: Memory for storing computer programs; A processor is used to execute the computer program stored in the memory, and when the computer program is executed, the image recognition data transmission method described in any one of claims 1 to 7 is implemented.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image recognition data transmission method described in any one of claims 1 to 7 is implemented.

Images