Image processing method, device, equipment and medium

Abstract

The present disclosure provides an image processing method, device, equipment, medium and program product, which can be applied to the field of information security technology and the field of finance. The image processing method includes: identifying an image to be processed, obtaining target state information and target position information, wherein the target state information includes the state information of a first object, the state information of a second object and the action state information of a third object; the target position information includes the position information of the first object, the position information of the second object and the position information of the third object; and when the target state information meets a preset condition, determining the behavior of the third object according to the spatial position relationship between the first object, the second object and the third object.

Description

Technical Field

[0001] The present disclosure relates to the field of information security technology, and in particular to an image processing method, apparatus, device, medium and program product. Background Art

[0002] The digitization of information provides convenience for the storage, access and display of information. With the help of computers, the required digital information can be efficiently stored and displayed. However, the security of digital information cannot be guaranteed and it is easy for information to be leaked through mobile storage devices, screen photography, etc.

[0003] As for the method of leaking information by taking pictures of the screen, since it does not require a physical connection to obtain the information displayed on the computer screen, it is highly concealed and not easy to be discovered. Summary of the invention

[0004] In view of the above problems, the present disclosure provides an image processing method, apparatus, device, medium and program product.

[0005] According to a first aspect of the present disclosure, there is provided an image processing method, comprising:

[0006] Identify the image to be processed, and obtain target state information and target position information, wherein the target state information includes state information of the first object, state information of the second object, and action state information of the third object; the target position information includes position information of the first object, position information of the second object, and position information of the third object; and

[0007] In the case where the target state information satisfies a preset condition, the behavior of the third object is determined according to the spatial position relationship among the first object, the second object and the third object.

[0008] According to an embodiment of the present disclosure, the state information of the first object includes any one of the following: mobile phone screen on state, mobile phone screen off state. The state information of the second object includes any one of the following: display on state, display off state. The action state information of the third object includes any one of the following: human head raised state, human head lowered state.

[0009] According to an embodiment of the present disclosure, the above-mentioned preset conditions include: the status information of the above-mentioned first object is the above-mentioned mobile phone screen is on state, the status information of the above-mentioned second object is the above-mentioned display is on state and the status information of the above-mentioned third object is the above-mentioned human head is raised state.

[0010] According to an embodiment of the present disclosure, the method for determining the behavior of the third object according to the spatial position relationship among the first object, the second object and the third object includes:

[0011] Determining a three-dimensional space region according to the position of the second object and the position of the third object;

[0012] The behavior of the third object is determined according to the spatial position relationship between the position of the first object and the three-dimensional space area.

[0013] According to an embodiment of the present disclosure, determining the three-dimensional space area according to the position of the second object and the position of the third object includes:

[0014] Using a second detection frame for marking the position of the second object and a third detection frame for marking the position of the third object as the bottom surface of the three-dimensional space area;

[0015] Each endpoint of the second detection frame is sequentially connected to each endpoint of a corresponding position of the third detection frame to obtain a three-dimensional space area.

[0016] According to an embodiment of the present disclosure, the step of determining the behavior of the third object according to the spatial position relationship between the position of the first object and the three-dimensional space region includes:

[0017] Calculating an intersection-over-union ratio between the three-dimensional space region and a first detection frame used to mark a position of the first object;

[0018] When it is determined that the intersection-over-union ratio of the first detection frame and the three-dimensional space area is greater than a preset threshold, the behavior of the third object is determined to be the target behavior.

[0019] According to an embodiment of the present disclosure, the above-mentioned identification of the image to be processed to obtain the target state information and the target position information includes: inputting the above-mentioned image to be processed into a target detection model to obtain the target state information and the target position information.

[0020] According to an embodiment of the present disclosure, the above target detection model is trained by the following operations:

[0021] Acquire sample data, wherein the sample data includes an image representing a mobile phone screen on state, an image representing a mobile phone screen off state, an image representing a display on state, an image representing a display off state, an image representing a human head raised, and an image representing a human head lowered;

[0022] The above sample data is input into the initial neural network model for training to obtain the target detection model.

[0023] The second aspect of the present disclosure provides an image processing device, including: an identification module and a determination module. The identification module is used to identify the image to be processed and obtain target state information and target position information, wherein the target state information includes the state information of the first object, the state information of the second object and the action state information of the third object; the target position information includes the position information of the first object, the position information of the second object and the position information of the third object. The determination module is used to determine the behavior of the third object according to the spatial position relationship between the first object, the second object and the third object when the target state information meets the preset conditions.

[0024] According to an embodiment of the present disclosure, the determination module includes a judgment unit and a determination unit. The judgment unit is used to judge whether the target state information satisfies a preset condition. The determination unit is used to determine the behavior of the third object according to the spatial position relationship between the first object, the second object and the third object when the target state information satisfies the preset condition.

[0025] According to an embodiment of the present disclosure, the determination unit includes a first determination submodule and a second determination submodule. The first determination submodule is used to determine the three-dimensional space region according to the position of the second object and the position of the third object. The second determination submodule is used to determine the behavior of the third object according to the spatial position relationship between the position of the first object and the three-dimensional space region.

[0026] According to an embodiment of the present disclosure, the first determination submodule includes a first determination subunit and a connection subunit. The first determination subunit is used to use the second detection frame for marking the position of the second object and the third detection frame for marking the position of the third object as the bottom surface of the three-dimensional space area. The connection subunit is used to connect each endpoint of the second detection frame with each endpoint of the corresponding position of the third detection frame in sequence to obtain the three-dimensional space area.

[0027] According to an embodiment of the present disclosure, the second determination submodule includes a calculation subunit and a second determination subunit. The calculation subunit is used to calculate the intersection-and-union ratio between the three-dimensional space area and the first detection frame used to mark the position of the first object. The second determination subunit determines that the behavior of the third object is the target behavior when it is determined that the intersection-and-union ratio between the first detection frame and the three-dimensional space area is greater than a preset threshold.

[0028] A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above-mentioned image processing method.

[0029] The fourth aspect of the present disclosure further provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the above-mentioned image processing method.

[0030] The fifth aspect of the present disclosure also provides a computer program product, including a computer program, which implements the above-mentioned image processing method when executed by a processor.

[0031] According to an embodiment of the present disclosure, by identifying the image to be processed, target state information and target position information are obtained. When it is determined that the target state information meets the preset conditions, a technical means is used to determine the behavior of the third object according to the spatial position relationship among the first object, the second object and the third object. Since the two factors of target state and target position are comprehensively considered, non-target behaviors of the third object can be filtered out, thereby reducing the misjudgment rate of the behavior of the third object. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0033] Figure 1 A diagram schematically illustrates an application scenario of an image processing method, apparatus, device, medium, and program product according to an embodiment of the present disclosure;

[0034] Figure 2 The flowchart of the image processing method according to the embodiment of the present disclosure is schematically shown;

[0035] Figure 3 A flowchart of a method for determining a third object behavior according to an embodiment of the present disclosure is schematically shown;

[0036] Figure 4 A flowchart of a method for determining a three-dimensional space region according to an embodiment of the present disclosure is schematically shown;

[0037] Figure 5 A schematic diagram schematically shows a three-dimensional space area and a first detection frame for marking a first object position according to an embodiment of the present disclosure;

[0038] Figure 6 The following schematically shows a logic block diagram of an image processing method according to an embodiment of the present disclosure;

[0039] Figure 7 A structural block diagram schematically shows an image processing device according to an embodiment of the present disclosure; and

[0040] Figure 8A block diagram of an electronic device suitable for implementing an image processing method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0041] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0042] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0043] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0044] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0045] Figure 1 The application scenario diagram of the image processing method according to the embodiment of the present disclosure is schematically shown.

[0046] like Figure 1 As shown, the application scenario 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0047] The user can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc. The terminal devices 101, 102, 103 can be electronic devices with a camera capable of acquiring images.

[0048] The terminal devices 101, 102, 103 may be various electronic devices with display screens capable of displaying acquired images, including but not limited to tablet computers, laptop computers, desktop computers, and the like.

[0049] The server 105 may be a server that provides various services, such as a background management server (only an example) that provides support for websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0050] It should be noted that the image processing method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the image processing device provided in the embodiment of the present disclosure can generally be set in the server 105. The image processing method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the image processing device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105.

[0051] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0052] It should be noted that the image processing method and device of the embodiment of the present disclosure can be used in the financial field, and can also be used in any field other than the financial field. The application field of the image processing method and device of the present disclosure is not limited.

[0053] In the technical solution disclosed in the present invention, the acquisition, storage and application of personal image information involved all comply with the provisions of relevant laws and regulations, take necessary confidentiality measures, and do not violate public order and good morals.

[0054] The following will be based on Figure 1 The scene described by Figure 2 to Figure 6 The image processing method of the disclosed embodiment is described in detail.

[0055] Figure 2The flowchart of the image processing method according to the embodiment of the present disclosure is schematically shown.

[0056] like Figure 2 As shown, the image processing of this embodiment includes operations S201 to S202.

[0057] In operation S201, the image to be processed is identified to obtain target state information and target position information. The target state information includes the state information of the first object, the state information of the second object and the action state information of the third object; the target position information includes the position information of the first object, the position information of the second object and the position information of the third object.

[0058] In operation S202, when the target state information satisfies a preset condition, a behavior of the third object is determined according to a spatial position relationship among the first object, the second object, and the third object.

[0059] According to an embodiment of the present disclosure, the first object may include a mobile device such as a mobile phone or a tablet computer for taking photos. The second object may include an electronic device such as a display for displaying information. The third object may be a person.

[0060] According to the embodiments of the present disclosure, by identifying the image to be processed, the target state information and the target position information are obtained. When the target state information meets the preset conditions, the technical means for determining the behavior of the third object is based on the spatial position relationship among the first object, the second object and the third object. The two factors of the target state and the target position are comprehensively considered, and the non-target behavior of the third object can be filtered out, thereby reducing the misjudgment rate of the behavior of the third object.

[0061] According to an embodiment of the present disclosure, the state information of the first object includes any one of the following: the mobile phone screen is on, the mobile phone screen is off. The state information of the second object includes any one of the following: the display is on, the display is off. The action state information of the third object includes any one of the following: the human head is raised, the human head is lowered.

[0062] According to an embodiment of the present disclosure, the preset conditions include: the status information of the first object is the mobile phone screen is on, the status information of the second object is the display is on, and the status information of the third object is the human head is raised.

[0063] According to the embodiments of the present disclosure, by determining the state information of the first object, the state information of the second object, and the action information of the third object, it can be determined whether the third object has the conditions to implement the behavior of shooting the screen. For example: when the mobile phone screen processing is turned on and the display is turned off, people cannot implement the behavior of shooting the screen. When the mobile phone screen processing is turned on and the display processing is turned on, although the conditions for implementing the behavior of shooting the screen are met, if the person does not take any action, the screen cannot be shot. Therefore, it is necessary to determine the state information of the first object, the state information of the second object, and the action state information of the third object in order to determine the behavior of the third object.

[0064] According to the embodiment of the present disclosure, by determining the status information of the first object, the status information of the second object and the action status information of the third object, it is possible to preliminarily determine whether the third object has the conditions to implement the target behavior, laying the foundation for further determining the behavior of the third object.

[0065] Figure 3 A flowchart of a method for determining a third object behavior according to an embodiment of the present disclosure is schematically shown.

[0066] like Figure 3 As shown, the method for determining the behavior of the third object in this embodiment includes operations S301-S302.

[0067] In operation S301 , a three-dimensional space area is determined according to a position of the second object and a position of the third object.

[0068] In operation S302, a behavior of a third object is determined according to a spatial position relationship between a position of the first object and the three-dimensional space region.

[0069] According to the embodiments of the present disclosure, the three-dimensional space area between the person and the display can be determined based on the positions of the display and the person in the image. For example, when the person sits directly opposite the display, the three-dimensional space area between the person and the display can be a hexahedral space between the plane where the human body is located and the plane where the display is located. At this time, the behavior of the third object can be determined by the placement of the mobile phone in this hexahedral space. For example, if the mobile phone is placed on the desktop where the display is placed, the screen of the mobile phone and the hexahedral space have no intersection or the intersection area is small, then it can be determined that the person is not shooting the screen. On the contrary, when the person stands the mobile phone upright, the intersection area between the screen of the mobile phone and the hexahedral space expands, and it can be determined that the person is shooting the screen.

[0070] According to an embodiment of the present disclosure, by determining a three-dimensional space region at the position of the second object and the position of the third object, and then determining the behavior of the third object based on the spatial position relationship between the first object and the three-dimensional space region, the misjudgment rate of the behavior of the third object can be reduced.

[0071] Figure 4 The flowchart of the method for determining a three-dimensional space area according to an embodiment of the present disclosure is schematically shown.

[0072] like Figure 4 As shown, the method for determining a three-dimensional space area in this embodiment includes operations S401 to S402.

[0073] In operation S401 , a second detection frame for marking a position of a second object and a third detection frame for marking a position of a third object are used as a bottom surface of a three-dimensional space region.

[0074] In operation S402, each endpoint of the second detection frame is sequentially connected to each endpoint of a corresponding position of the third detection frame to obtain a three-dimensional space region.

[0075] Figure 5 A schematic diagram of a three-dimensional space area and a first detection frame for marking a position of a first object according to an embodiment of the present disclosure is schematically shown.

[0076] like Figure 5 As shown, according to an embodiment of the present disclosure, for example, a person sits upright in front of a display, the detection frame used to mark the position of the display may be a second rectangular frame 502, and the detection frame used to mark the position of the human head may be a third rectangular frame 503. Since the person sits upright in front of the display, the second rectangular frame is parallel to the third rectangular frame. The endpoints A1, A2, A3, A4 of the second rectangular frame 502 are connected with the endpoints B1, B2, B3, B4 of the third rectangular frame 503 in sequence to obtain a three-dimensional space area 504.

[0077] According to the embodiment of the present disclosure, a three-dimensional space area is established with the detection box marking the positions of the second object and the third object as the bottom surface, and a behavior area is demarcated in which the third object can operate the real-time behavior of the first object. By determining the behavior of the third object within the behavior area, the effective range of the judgment behavior can be determined, thereby reducing the misjudgment rate.

[0078] According to an embodiment of the present disclosure, determining the behavior of the third object according to the spatial position relationship between the position of the first object and the three-dimensional space region includes:

[0079] An intersection-over-union ratio between the three-dimensional space region and a first detection box for marking a position of the first object is calculated.

[0080] When it is determined that the intersection-over-union ratio of the first detection frame and the three-dimensional space area is greater than a preset threshold, the behavior of the third object is determined to be the target behavior.

[0081] According to the embodiment of the present disclosure, Figure 5 As shown, the detection frame used to mark the location of the mobile phone can be the first rectangular frame 501. For example, by calculating the ratio of the area of ​​the three-dimensional space region 504 to the area of ​​the third rectangular frame 501 (intersection-over-combination ratio) to be 1, it is considered that the area of ​​the three-dimensional space region 504 is equal to that of the third rectangular frame 501. In the case where the preset threshold range is 0 to 0.8, since the calculated intersection-over-combination ratio exceeds the preset threshold range, it can be determined that the mobile phone is facing the display, and it is determined that the person is performing a behavior similar to shooting the screen.

[0082] According to the embodiment of the present disclosure, the behavior of the third object is determined by determining whether the intersection-and-union ratio of the first detection frame and the three-dimensional space area exceeds a preset threshold, and the spatial position relationship is digitized, so that automated judgment and processing can be achieved.

[0083] According to an embodiment of the present disclosure, identifying an image to be processed and obtaining target state information and target position information includes: inputting the image to be processed into a target detection model to obtain the target state information and target position information.

[0084] According to the disclosed embodiment, the target detection model can adopt a neural network model. The neural network model can adopt the target detection algorithm Yolov5, which improves the calculation speed and accuracy by performing mosaic data enhancement, adaptive anchor frame calculation, adaptive image scaling, and setting a focus structure and a CSP (Chip Scale Package) chip-scale packaging structure in the input model training stage, adding an FPN (Feature Pyramid Network) feature pyramid network structure and a PAN (Personal Area Network) personal area network structure, and improving the loss function during training and the prediction box screening function.

[0085] According to the embodiments of the present disclosure, by identifying target state information and target position information through a target detection model, the target can be identified quickly and accurately to facilitate subsequent determination steps and improve processing speed.

[0086] According to an embodiment of the present disclosure, the target detection model is trained through the following operations.

[0087] Acquire sample data, wherein the sample data includes an image representing a mobile phone screen on state, an image representing a mobile phone screen off state, an image representing a display on state, an image representing a display off state, an image representing a human head raised, and an image representing a human head lowered;

[0088] The sample data is input into the initial neural network model for training to obtain the target detection model.

[0089] According to the embodiments of the present disclosure, the sample data may include images captured by a camera. For example, an image representing the on-screen state of a mobile phone may be an image of a single on-screen state of the mobile phone, an image including the on-screen state of the mobile phone and the off-screen state of the display, an image including the on-screen state of the mobile phone and the lowered head of a human body, and the like.

[0090] According to an embodiment of the present disclosure, the initial neural network model may select a convolutional neural network model, a recurrent neural network model, and the like.

[0091] According to an embodiment of the present disclosure, an object detection model is trained by using images that can represent the state of the target as sample data, so that the object detection model can identify the state of the target in the image, such as whether the mobile phone screen is turned on, whether the display screen is turned on, etc., so as to determine whether a person can perform the action of shooting the screen.

[0092] Figure 6 The logical block diagram of the image processing method according to the embodiment of the present disclosure is schematically shown.

[0093] like Figure 6 As shown, the image processing method of this embodiment may include the following operations.

[0094] First, identify the image to be processed and obtain the target state information and target position information. Then, determine whether the target state information meets the preset conditions. If it is determined that the target state information does not meet the preset conditions, no processing is performed. If it is determined that the target state information meets the preset conditions, the three-dimensional space area is determined according to the position of the second object and the position of the third object, and then the intersection and union ratio between the three-dimensional space area and the first detection frame used to mark the position of the first object is calculated to determine whether the intersection and union ratio exceeds the threshold range. If it does not exceed the threshold range, it is determined that the behavior of the third object is not the target behavior. If it exceeds the threshold range, it is determined that the behavior of the third object is the target behavior.

[0095] According to the embodiments of the present disclosure, for example: identify the image to be processed, and obtain the target state information and the target position information. The target state information is as follows: the state of the first object mobile phone is the mobile phone screen on state, the state of the second object display is the display screen on state, and the dynamic state of the third object is the human head raised state. After judgment, if the target state information meets the preset conditions, the three-dimensional space area is determined according to the position of the display and the position of the human head. It is assumed that the intersection-and-union ratio between the three-dimensional space area and the first detection frame used to mark the position of the mobile phone is 1, and the threshold range is set to 0 to 0.5. If the intersection-and-union ratio exceeds the threshold range, it is determined that the behavior of the third object person is the screen shooting behavior.

[0096] Based on the above image processing method, the present disclosure also provides an image processing device. Figure 7 The device is described in detail.

[0097] Figure 7 The structure block diagram of the image processing device according to the embodiment of the present disclosure is schematically shown.

[0098] like Figure 7 As shown, the image processing device 800 of this embodiment includes a recognition module 701 and a determination module 702 .

[0099] The recognition module 701 is used to recognize the image to be processed and obtain target state information and target position information, wherein the target state information includes the state information of the first object, the state information of the second object and the action state information of the third object; the target position information includes the position information of the first object, the position information of the second object and the position information of the third object. In one embodiment, the recognition module 701 can be used to perform the operation S201 described above, which will not be repeated here.

[0100] The determination module 702 is used to determine the behavior of the third object according to the spatial position relationship between the first object, the second object and the third object when the target state information meets the preset condition. In one embodiment, the determination module 702 can be used to perform the operation S202 described above, which will not be repeated here.

[0101] According to an embodiment of the present disclosure, the determination module 701 includes a judgment unit and a determination unit. The judgment unit is used to judge whether the target state information satisfies a preset condition. The determination unit is used to determine the behavior of the third object according to the spatial position relationship between the first object, the second object and the third object when the target state information satisfies the preset condition.

[0102] According to an embodiment of the present disclosure, the determination unit includes a first determination submodule and a second determination submodule. The first determination submodule is used to determine the three-dimensional space region according to the position of the second object and the position of the third object. The second determination submodule is used to determine the behavior of the third object according to the spatial position relationship between the position of the first object and the three-dimensional space region.

[0103] According to an embodiment of the present disclosure, the first determination submodule includes a first determination subunit and a connection subunit. The first determination subunit is used to use the second detection frame for marking the position of the second object and the third detection frame for marking the position of the third object as the bottom surface of the three-dimensional space area. The connection subunit is used to connect each endpoint of the second detection frame with each endpoint of the corresponding position of the third detection frame in sequence to obtain the three-dimensional space area.

[0104] According to an embodiment of the present disclosure, the second determination submodule includes a calculation subunit and a second determination subunit. The calculation subunit is used to calculate the intersection-and-union ratio between the three-dimensional space area and the first detection frame used to mark the position of the first object. The second determination subunit determines that the behavior of the third object is the target behavior when it is determined that the intersection-and-union ratio between the first detection frame and the three-dimensional space area is greater than a preset threshold.

[0105] According to an embodiment of the present disclosure, any multiple modules in the identification module 701 and the determination module 702 can be combined in one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the identification module 701 and the determination module 702 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the identification module 701 and the determination module 702 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0106] Figure 8 A block diagram of an electronic device suitable for implementing an image processing method according to an embodiment of the present disclosure is schematically shown.

[0107] like Figure 8As shown, the electronic device 800 according to an embodiment of the present disclosure includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 808 to a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include an onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0108] In RAM 803, various programs and data required for the operation of electronic device 800 are stored. Processor 801, ROM 802 and RAM 803 are connected to each other via bus 804. Processor 801 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 802 and / or RAM 803. It should be noted that the program can also be stored in one or more memories other than ROM 802 and RAM 803. Processor 801 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0109] According to an embodiment of the present disclosure, the electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to the bus 804. The electronic device 800 may further include one or more of the following components connected to the I / O interface 805: an input portion 806 including a keyboard, a mouse, etc.; an output portion 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 808 including a hard disk, etc.; and a communication portion 809 including a network interface card such as a LAN card, a modem, etc. The communication portion 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that a computer program read therefrom is installed into the storage portion 808 as needed.

[0110] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0111] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 802 and / or RAM 803 described above and / or one or more memories other than ROM 802 and RAM 803.

[0112] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the item recommendation method provided by the embodiment of the present disclosure.

[0113] The above functions defined in the system / device of the embodiment of the present disclosure are executed when the computer program is executed by the processor 801. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0114] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 809, and / or installed from a removable medium 811. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0115] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the processor 801, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0116] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0117] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0118] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0119] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. An image processing method, comprising: Identify the image to be processed, and obtain target state information and target position information, wherein the target state information includes state information of a first object, state information of a second object, and action state information of a third object; the target position information includes position information of the first object, position information of the second object, and position information of the third object; and When the target state information satisfies a preset condition, determining a behavior of the third object according to a spatial position relationship among the first object, the second object and the third object; Wherein, determining the behavior of the third object according to the spatial position relationship among the first object, the second object and the third object includes: Calculating an intersection-over-union ratio between a three-dimensional space region and a first detection frame used to mark the position of the first object; wherein the three-dimensional space region is established with the detection frame used to mark the positions of the second object and the third object as a bottom surface; When it is determined that the intersection-over-union ratio of the first detection frame and the three-dimensional space area is greater than a preset threshold, the behavior of the third object is determined to be the target behavior.

2. The method according to claim 1, wherein: The state information of the first object includes any one of the following: mobile phone screen on state, mobile phone screen off state; The state information of the second object includes any one of the following: display on state, display off state; The action state information of the third object includes any one of the following: a state where the human head is raised and a state where the human head is lowered.

3. The method according to claim 2, wherein: The preset conditions include: The status information of the first object is the on-state of the mobile phone screen, the status information of the second object is the on-state of the display, and the status information of the third object is the raised state of the human head.

4. The method according to claim 1, wherein: The three-dimensional space region is established with a detection frame for marking the positions of the second object and the third object as a bottom surface, and includes: using a second detection frame for marking the position of the second object and a third detection frame for marking the position of the third object as the bottom surface of the three-dimensional space area; Each endpoint of the second detection frame is sequentially connected to each endpoint of a corresponding position of the third detection frame to obtain a three-dimensional space area.

5. The method according to claim 1, wherein: The step of identifying the image to be processed and obtaining target state information and target position information includes: The image to be processed is input into a target detection model to obtain target state information and target position information.

6. The method according to claim 5, wherein: The object detection model is trained by: Acquire sample data, wherein the sample data includes an image representing a mobile phone screen on state, an image representing a mobile phone screen off state, an image representing a display on state, an image representing a display off state, an image representing a human head raised, and an image representing a human head lowered; The sample data is input into an initial neural network model for training to obtain a target detection model.

7. An image processing device, comprising: an identification module, used to identify the image to be processed, and obtain target state information and target position information, wherein the target state information includes state information of the first object, state information of the second object, and action state information of the third object; the target position information includes position information of the first object, position information of the second object, and position information of the third object; a determination module, configured to determine a behavior of the third object according to a spatial position relationship among the first object, the second object and the third object when the target state information satisfies a preset condition; Wherein, the determination module includes: a calculation subunit, configured to calculate an intersection-over-union ratio between a three-dimensional space region and a first detection frame used to mark the position of the first object; wherein the three-dimensional space region is established with the detection frame used to mark the positions of the second object and the third object as a bottom surface; The second determination subunit is used to determine that the behavior of the third object is the target behavior when it is determined that the intersection-over-union ratio of the first detection frame and the three-dimensional space area is greater than a preset threshold.

8. A computer system comprising: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having executable instructions stored thereon, wherein when the instructions are executed by a processor, the processor implements the method according to any one of claims 1 to 6.

10. A computer program product comprising: Computer executable instructions, when executed, are used to implement the method according to any one of claims 1 to 6.

Images